Dear SIGMOD 2009 author,
In case you volunteer to participate in the Repeatability &
Workability Evaluation process for accepted SIGMOD 2009 papers, we
kindly ask you to prepare the following and make it available for us to
download, preferably by Wednesday April 22 2009.
- Camera-ready version of your accepted SIGMOD 2009 paper (as PDF).
- Your code plus scripts and instruction to compile it on (at least) the
same platform (hardware architecture and operating system) as you used
(see details below).
In addition to the code, you can also provide pre-compiled binaries.
In the exceptional case that your cannot provide us (RWE + RWC) with your
code for good reasons (i.e., strict commercial IP limitation --- please
explain in detail!), supplying only your binaries would be acceptable.
- Scripts/programs to run your experiments, extract the data as used in your
papers and present the results as tables, graphs, etc. as in your paper.
Please make sure, you document your scripts/programs well and include
detailed usage instructions.
Describe each experiment individually and relate it to the respective
section and/or table/figure in your paper.
For the "workability" evaluation, please also provide scripts/programs for
alternative and/or additional experiments that go beyond the ones
presented in your paper, or at least include suggestions how to modify
and/or extend your experiments by using alternative datasets and/or
different parameters.
- The datasets (and/or data generators) you used for your experiments plus
preferably alternative/extended/modified datasets for extended
experimentation for the "workability" evaluation. Pointers to the original
sources of publicly available datasets (and/or generators) are also
acceptable.
- Specifications of the computer system(s) required to run your experiments,
including the following information:
+ Operating System
(e.g., "Windows Vista 32bit", "Fedora 9 (Linux 2.6.25)",
"MacOSX 10.5 Leopard (Darwin 9.5.0)", etc.)
+ Class of hardware
(e.g., Laptop, Desktop, Workstation, Server, ...)
+ CPU type
(e.g., "Intel Core2 Quad CPU Q6600 @ 2.40GHz",
"Intel Pentium M (Dothan) @ 1.50GHz", etc.)
+ # cores per CPU
+ # CPUs per machine
+ main memory (RAM) size per machine
+ storage/disk system
(e.g. "single 120 GB laptop disk @ 5400 RPM",
"5x 500 GB S-ATA disk at 7200 RPM as RAID-0", "32 GB SSD", etc.)
+ free disk space per machine
- Information about required software, like DBMSs, compilers, scripting
languages, virtual machines, compilers, tools, etc.
The following DTD and XML example should help you to collect and organize
this information in a "standardized" format:
SIGMOD-2009-RWE.dtd
SIGMOD-2009-RWE.xml
Preferably, create a clear directory structure to collect the above listed
collections of files, and pack everything into a portable archive (e.g.,
zip, tar).
In case you have any comments and/or questions regarding the SIGMOD 2009
Repeatability & Workability Evaluation process, please do not hesitate to
contact us.
We will provide you with further details about Wiki/Blog for interaction
with the reviewers within the next two weeks.
Thank you very much in advance for your cooperation.
Best regards,
Ioana Manolescu
Stefan Manegold
--
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |