- We will run the submitted programs on the system-level (as opposed to language-level or application-level) virtual machine KVM with QEMU.
- The host operating system will be Debian GNU/Linux 5.0 (lenny) for amd64. The guest will be Debian GNU/Linux 6.0 (squeeze) for amd64. The host OS is older than the guest for administrative reasons. For the same reasons, we plan to use the version of KVM/QEMU in the "kvm" (not "qemu") package of lenny.
- The host (physical) machines will be a cluster of Intel
Xeon E5530 2.4 GHz CPUs (Intel VT enabled) with 24 GB main memory each (or another cluster with similar hardware, depending on availability) . Each guest (virtual) machine will have a single-core CPU provided as the default of KVM/QEMU, 1 GB main memory with 2 GB swap and 30 GB (virtual) disk space. - Each submitted program will be run by a unique general user (as opposed to a shared or privileged account) on the guest OS. We plan to impose the hard limit
sof512 MB virtual memory and1 GB disk usage on each submission. [Update on May 27: For technical reasons, we will actually impose the hard limits of 8 MB stack (the default of Debian squeeze) and 512 MB data segment, and no limit for virtual memory (address space used), on each process. However, any program whose total memory footprint (the sum of the resident set sizes of all of your processes running simultaneously) exceeds 512 MB at runtime (as opposed to installation time) may be disqualified by the judges. Usually, one can check it by watching the "RES" field of the "top" command.] There will also be a hard limit for CPU usage, which will be announced in the task description when the contest starts. No network connection will be available for the submitted programs at runtime. - We accept requests for installing standard Debian squeeze packages (those listed in http://packages.debian.org/squeeze/) required for running (not compiling) your program, from now on until the contest ends. Please specify the exact name(s) of the package(s) in a comment to this post (do not e-mail to the organizers or comment to other posts for this purpose). Please also check the list of already installed packages to make sure that your request is not subsumed yet.
- If the runtime/libraries/files required for running your program is not available as a standard package, submit them together with your program. We will install your submission in the home directory of a unique general user. [Update on May 15: If your program needs to know the absolute path of this directory, please try to use $HOME or something similar. In case this is impossible, e-mail to icfpc2011-blogger AT kb.ecei.tohoku.ac.jp and request a specific username (please try to choose a unique one).] Details of the submission procedure will be described in the contest task.
- In either case, we will do our best to run each submitted program. For this purpose, we may contact you (by e-mail to the address registered at the time of submission) after the contest has ended.
- Source code of your program must also be submitted, though it will not be compiled by the judges (except for the winners, whose submissions will be examined with more attention).
- We will not distribute the image of our virtual machine because of network bandwidth limitations. It should anyway be easy for oneself to install Debian squeeze on a virtual (or physical) machine.
[The ICFP Programming Contest 2011 organizers, with special thanks to the InTrigger team]
[Update on June 13: It would be the best if each participant can prepare an environment (virtual or physical) with Debian squeeze for amd64 (which supports x86-64 including Intel 64), but in case you can only produce 32-bit executables, please remember:
- Statically linked 32-bit Linux executables should run with no problem on the 64-bit environment;
- Dynamically linked 32-bit libraries should also be fine as long as they are included in the ia32-libs package of Debian; and
- Other dynamically linked 32-bit libraries (and any other files required at runtime but unavailable on Debian squeeze for amd64) can be included in the submission itself.]
I would respectfully like to register my disappointment in the fact that the judge's test system will only have access to a single CPU core. The ability to safely and efficiently construct concurrent solutions has been a shining light of functional programming. To hobble this ability with the use of only a single core seem to me to be against the spirit of the IFPC Programming Contest's aims. I ask that the judges reconsider this limitation.
ReplyDeleteUnfortunately, it is a limitation of the system we have access to. Thank you for your understanding.
ReplyDeleteNot knowing the nature of the task (interpreted languages may be totally inappropriate), but can you add package "ruby1.9.1", please?
ReplyDeleteTeam taxidriver
Request granted and list updated:
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
re vm image distribution: wouldn't it be possible to set up a torrent (as a way to limit bandwidth used)?
ReplyDeleteUnfortunately, many universities restrict P2P communication nowadays, including ours. It is anyway very easy (in our experience, even easier than downloading a VM image!) to install Debian squeeze on VirtualBox, VMware, etc.
ReplyDelete>> We plan to impose the hard limits of 512 MB virtual memory
ReplyDeleteDoes this mean the virtual machine will only have 512 MB of ram or that our process will be monitored and killed if it goes over 512 MB allocated?
Will we be able to know the full pathname of the directory where our program is installed, so that we can easily hard-code that path to the executables we build?
ReplyDeleteI presume this could be implemented easily by each team automatically getting a unique short alphanumeric id when they send their first sumbission, and the directory being a function of that id.
Please try to use $HOME or something similar. In case this is impossible, e-mail to icfpc2011-blogger AT kb.ecei.tohoku.ac.jp and request a specific username (please try to choose a unique one).
ReplyDelete> Does this mean the virtual machine will only have 512 MB of ram or that our process will be monitored and killed if it goes over 512 MB allocated?
ReplyDeleteThe latter. As specified in the original post, the virtual machine has more memory.
Will the test machines have ia32-libs on it and be able to run binaries compiled for 32 bits?
ReplyDeletePlease provide the llvm and llvm-dev packages.
ReplyDeleteThanks,
The higher order of zeuxis
No Erlang packages available?
ReplyDeleteAs requested, packages "ia32-libs", "llvm", "llvm-dev", and "erlang" have been installed:
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
As stated in the original post, please specify the exact names of the Debian squeeze packages you request (rather than asking "is XXX installed?").
Please install Python and Psyco on the machine.
ReplyDelete> Please install Python and Psyco on the machine.
ReplyDeleteAs stated in the original post (and the previous comment), please specify the exact names of the Debian squeeze packages:
http://packages.debian.org/squeeze/
and please also check the list of already installed packages:
http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Please add:
ReplyDeletescala (2.7.7.dfsg-4)
scala-library (2.7.7.dfsg-4)
Thanks!
Packages "scala" and "scala-library" are now installed:
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Packages requested:
ReplyDeletemlton
smlnj
libsmlnj-smlnj
liblua5.1-0
liblua5.1-0-dbg
liblua5.1-0-dev
liblua5.1-copas-dev
liblua5.1-copas0
liblua5.1-doc0
liblua5.1-expat-dev
liblua5.1-expat0
liblua5.1-filesystem-dev
liblua5.1-filesystem0
liblua5.1-json
liblua5.1-logging
liblua5.1-logging-dev
liblua5.1-lpeg-dev
liblua5.1-lpeg2
liblua5.1-md5-0
liblua5.1-md5-dev
liblua5.1-posix-dev
liblua5.1-posix1
liblua5.1-socket-dev
liblua5.1-socket2
liblua5.1-sql-sqlite2
lua5.1
Installed, except for two unfound packages "smlnj" and "libsmlnj-smlnj". I am afraid they are not included in the standard distribution of Debian squeeze. Please include the necessary runtime in your own submission.
ReplyDeleteIs possible to get libmono-system-runtime2.0-cil ?
ReplyDeleteSure, of course!
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
pari-gp
ReplyDeletepari-extra
pari-gp2c
Thanks!
How about gnu-smalltalk and gforth ?
ReplyDeleteInstalled.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Please install
ReplyDeletesun-java6-bin (6.24-1~squeeze1) [non-free]
sun-java6-jdk (6.24-1~squeeze1) [non-free]
sun-java6-jre (6.24-1~squeeze1) [non-free]
and remove
openjdk-6-jre
openjdk-6-jre-headless
openjdk-6-jre-lib
gcj-4.4-base
gcj-4.4-jre
gcj-4.4-jre-headless
gcj-4.4-jre-lib
gcj-jre
gcj-jre-headless
OpenJDK is not portable enough and not well-tested (compared to Sun's reference implementation). Thus people writing in Java/JVM-based languages wouldn't be able to develop on a different platform and be sure that it actually runs correctly on OpenJDK.
GCJ is know to break a lot of things in Java and it shouldn't be used. When GCJ is installed it's often tries to be "the" java, making conflicts for the software which tries to use the real Java. Please remove it (and all the packages that depend on GCJ specifically, like openoffice.org-gcj, bash-gcj and some libraries that have gcj in them).
Please install 'sun-java6-jdk' and its dependencies.
ReplyDeleteGranted.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Please,
ReplyDeletesbcl clisp
Sure.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Please install python-psyco (http://packages.debian.org/en/squeeze/python/python-psyco)
ReplyDelete>> our process will be monitored and killed if it goes over 512 MB allocated?
ReplyDelete> The latter
Can you please review this? Many runtimes used to overcommit memory. For example, right now I ran nearly empty java process and VSZ goes as high as 650M. And sometimes it is very hard to change.
We have checked that Java (Sun JRE 1.6.0_24 with the default options) runs under the 512MB virtual memory use hard limit on judges' environment. If there is a real problem with this limit, please let us know again.
ReplyDeleteIt was:
ReplyDeleteopenjdk-6-jre 6b18-1.8.3-2+squeeze1
clojure 1.1.0+dfsg-1
$ clojure
VSZ=679708
I did not try sun jre
then,
sbcl 1:1.0.47.0-2
$ sbcl
VSZ=538156
That all was debian 32bit.
Please install:
ReplyDeleteocaml-nox
ocaml-findlib
camlp4-extra
ocaml-native-compilers
libextlib-ocaml-dev
libcore-extended-ocaml-dev
libcore-ocaml-dev
liblwt-ocaml-dev
libbatteries-ocaml-dev
libocamlgraph-ocaml-dev
libpcre-ocaml-dev
libbitstring-ocaml-dev
Thx
You say that we need to submit the compiled code and the source code is not enough.
ReplyDeleteWhat if I submit the source code (say C++) and a script (say bash) compiling and then running the code?
please add haskell-platform
ReplyDeleteThank you, max630. We will consider what we can do, and will respond to your request as soon as possible.
ReplyDeleteI am afraid that "python-psyco" is not available as a standard Debian squeeze package for amd64. If necessary, please consider including its runtime (if any) in your submission.
ReplyDelete> What if I submit the source code (say C++) and a script (say bash) compiling and then running the code?
ReplyDeleteThat is OK, but there would be a risk that the compilation script fails for some reason. Please avoid that as far as possible.
Requested packages (related to OCaml and Haskell) have been installed:
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
In response to previous comments (by max630) regarding the hard limit on virtual memory use, we have updated our specification of the environment. Please see "Update on May 27".
ReplyDeletePython 3 and PLT Scheme, please :)
ReplyDeletepython3
plt-scheme
http://packages.debian.org/squeeze/python/python3
http://packages.debian.org/squeeze/lisp/plt-scheme
Hello,
ReplyDeleteplease install the package
groovy
Best,
Maxym
please disregard my previous comment.
ReplyDeleteDebian's groovy package is unfortunately too old to be usable.
Sorry, Maxym
> Python 3 and PLT Scheme, please :)
ReplyDeleteAcknowledged.
http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
> Debian's groovy package is unfortunately too old to be usable.
We are sorry about the limitations of our environment - our manpower is finite and we had to draw the line somewhere.
Why wouldn't you put vm image on some public tracker?
ReplyDeleteDo it from home if university torrent policy is so strict.
Do you have a more specific suggestion? The many gigabytes of VM image resides on a university cluster and is updated upon every installation request. (Please also understand that, although we are doing our best, we are volunteers and doing these in our spare time.)
ReplyDeleteAnd once again, installing Debian squeeze on your VM would be _much_ faster and easier than downloading the VM image. Please just try if you doubt.:-) Thank you!
Hello,
ReplyDeleteplease install the package
python-gmpy
Best,
Sergey
With pleasure.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
For what it's worth, I'm using the following very simple script to keep my VirtualBox-base Debian squeeze VM up-to-date (apologies for the bad formatting):
ReplyDeletewget -N 'http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt'
sudo apt-get install $(
cat apt-list.txt \
| sed -e 's!^....\([^ ]*\).*!\1!' \
| grep -v -E '^(xserver-xorg-video-siliconmotio)$'
)
I need to filter out one of the xserver-xorg packages, which cannot be installed for some reason: apt-get gives me "E: Unable to locate package xserver-xorg-video-siliconmotio". (And this is not because of the chopped off package name.) But this is not a package one would require, I imagine. Although, who knows... :-)
chicken-bin please
ReplyDeleteACK.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Hello,
ReplyDeleteI've never installed a linux distribution on VMWare. Could you please provide a link explaining how it is done, if possible specific to the distribution you require?
Thanks,
Alan
Could you please include Gauche in the environment?
ReplyDeletegauche (0.9-16)
gauche-c-wrapper (0.6.1-4)
gauche-dev (0.9-16)
gauche-gdbm (0.9-16)
gauche-gl (0.4.4-5)
gauche-gtk (0.4.1-13)
gauche-zlib (0.9-16)
> I've never installed a linux distribution on VMWare. Could you please provide a link explaining how it is done, if possible specific to the distribution you require?
ReplyDeleteWe are sorry but we do not have sufficient manpower to provide technical support for basic software. Please read the manuals and/or search the WWW (or just try and see - most if not all of the installation procedures are self-explanatory).
> Could you please include Gauche in the environment?
ReplyDeleteDone.
http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
spidermonkey-bin please.
ReplyDeleteGot it.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
libmath-gmp-perl
ReplyDeletelibmath-bigint-gmp-perl
libanyevent-perl
libarchive-zip-perl
libmodule-install-perl
libdbd-sqlite3-perl
libdbi-perl
libev-dev
libev3
liblist-moreutils-perl
libreadonly-xs-perl
libreadonly-perl
Roger.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
zsh please
ReplyDeleteJust to clarify, we'll need to submit binaries for amd64? Does this differ from x86-64? I'm afraid I can't find among our team any hardware specifically with an AMD 64-bit CPU in it.
ReplyDelete> zsh please
ReplyDeleteHere you are.
http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
> Just to clarify, we'll need to submit binaries for amd64? Does this differ from x86-64? I'm afraid I can't find among our team any hardware specifically with an AMD 64-bit CPU in it.
ReplyDeleteAny executable is fine as long as it can be run on our guest environment. For more information on Debian and amd64 (and perhaps some virtual machine software), please read the documents and/or search the WWW, for example:
http://www.debian.org/ports/amd64/index.en.html
> The port consists of a kernel for all AMD 64bit CPUs with AMD64 extension and all Intel CPUs with EM64T extension, and a common 64bit userspace.
Speaking of amd64, would a binary created in a VM running the i386 version of Squeeze work on your system?
ReplyDeleteAgain, we are sorry but we cannot provide technical support for individual languages. Please prepare a compatible environment (on a virtual machine if necessary) and check by yourself.
ReplyDeleteP.S. See also (again):
ReplyDeletehttp://www.debian.org/ports/amd64/index.en.html
> Native execution of legacy 32bit binaries is supported by the kernel, and core libraries are provided by the ia32-libs package.
However, this mainly applies to C (and C++) libraries only. As we wrote above, we respectfully ask you to test your language on an environment compatible to ours (again on a virtual machine if necessary) well in advance.
I guess we're not understanding each other: I want to know if installing the i386 version of Squeeze in a VM is a "compatible environment". (I'm installing on VMWare on a MacBook Pro with a Core 2 duo processor.)
ReplyDeleteTo rephrase: how can I test if my environment is compatible?
Thanks,
Alan
> I want to know if installing the i386 version of Squeeze in a VM is a "compatible environment"
ReplyDeleteThe short general answer is no, of course, because i386 is a different architecture (32-bit) from amd64 (64-bit). In particular, some dynamically linked 32-bit libraries may not be available on a 64-bit environment (though some 32-bit "core libraries" are provided as the "ia32-libs" package in Debian).
Just in case you are not familiar with the terminology, both AMD64 and Intel 64 are implementations of x86-64 and are compatible with each other (modulo a few minor differences which are unimportant in most operating systems). A Google search for "amd64" gives the following page at the top.
http://en.wikipedia.org/wiki/X86-64
> To rephrase: how can I test if my environment is compatible?
Please just install Debian squeeze for amd64 on your VMware. It is also compatible with Core 2 Duo. Once again, a Google search for "Debian amd64" gives at the top:
http://www.debian.org/ports/amd64/index.en.html
I hope these help (and please read the documents and/or search the WWW by yourself before asking us questions about general hardware and software - thank you very much for your understanding and cooperation!).
I had found this last page, which does not mention core 2 duo by the way, and my question was: is the included support for i386 enough, which you just answered. Thanks a lot! And one last clarification: these were not "general" hardware and software questions, they were quite specific ones about the very specific requirements of this contest.
ReplyDeleteThanks for the clarifications. It is the best to prepare a Debian squeeze environment for amd64 (virtual or physical), but just in case you can only produce 32-bit executables, please remember:
ReplyDelete- Statically linked 32-bit Linux executables should run with no problem on the 64-bit environment.
- Dynamically linked 32-bit libraries should also be fine as long as they are included in the ia32-libs package of Debian.
- Other dynamically linked 32-bit libraries (and any other files required at runtime and not available on Debian squeeze for amd64) can be included in the submission itself.
I am sorry for my poor English but all of these are direct consequences of readily available information on the Web. We truly appreciate your cooperation. Thank you very much!
Eijiro
Could you clarify whether OpenJDK and GCJ are removed as requested by Ivan T?
ReplyDeleteYes, they have been removed.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Could you please install:
ReplyDeletephp5-cli
Thanks!
Would you please add packages flex and bison?
ReplyDelete> php5-cli
ReplyDelete> Would you please add packages flex and bison?
Processed.
http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
minisat2
ReplyDeletepicosat
Thanks - J.W.
You're welcome.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt
Is it possible to get the output of dpkg --get-selections rather than the apt-list.txt file you have? I don't know if it's possible to automate installations from the file you've provided, but from the get-selections output it's trivial to ensure all available packages are installed in my development image: http://lists.samba.org/archive/linux/2002-June/003399.html
ReplyDeleteThank you for the suggestion!
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt
See also:
http://www.icfpcontest.org/2011/05/judges-machine-and-system-environment.html?showComment=1307110031039#c1196844878145834435
We will keep both lists up-to-date.
Awesome, thanks!
ReplyDeletePackage request:
ReplyDeletemono-complete
I know this isnt an official debian package, but its possible to have the f# package from http://fsxplat.codeplex.com/
ReplyDelete?
thanks anyway.
Will you add
ReplyDeleteemacs23
to the list, so that I (and other emacs users) can edit our code on the VM?
I guess I can install emacs by myself and the difference between the judge's machine and mine will bring no inconsistency, but just in case.
> I know this isnt an official debian package, but its possible to have the f# package from http://fsxplat.codeplex.com/ ?
ReplyDeleteWe are sorry but we cannot install non-official packages. Please include in your submission whatever is necessary for _running_ (not _compiling_) your program.
> mono-complete
ReplyDelete> emacs23
Installed.
http://www.kb.ecei.tohoku.ac.jp/icfpc/apt-list.txt (we move this file but the previous address will also remain valid)
http://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt
You already konw this but just to make sure for others, participants do not have to _develop_ their programs on the VM; it is just for _testing_ their submissions.
Could you please install libneko0 package?
ReplyDeleteCould you please install "neko" package? I'm not sure whether libneko0 will be enough.
ReplyDelete> libneko0
ReplyDelete> libneko
Installed to the unofficial duel server.
Will soon be installed to the official machine image, too.
fp-* and fpc-* packages aka FreePascal thingies please. :)
ReplyDelete> fp-compiler
ReplyDelete> fp-ide
> fp-units-base
> fp-units-db
> fp-units-fcl
> fp-units-fv
> fp-units-gfx
> fp-units-gnome1
> fp-units-gtk
> fp-units-gtk2
> fp-units-math
> fp-units-misc
> fp-units-multimedia
> fp-units-net
> fp-units-rtl
> fp-utils
> fpc
are installed to the unofficial server.
Will also be included on the official machine.
Hi everybody, will you please install python-dev [1] ?
ReplyDeletethanks in advance
sciamp
[1] http://packages.debian.org/squeeze/python-dev
> python-dev
ReplyDeleteDone.
Due to a number of requests and technical reasons, we have installed the F# package available from:
ReplyDeletehttp://fsxplat.codeplex.com/releases/view/48831
Hi, can you please install libboost-program-options-dev?
ReplyDeleteThanks in advance!
> libboost-program-options-dev
ReplyDeleteProcessed.
By request, we have installed Ruby 1.8 packages.
ReplyDeletehttp://www.kb.ecei.tohoku.ac.jp/icfpc/apt-list.txt (we moved this file but the previous address will also remain valid)
http://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt
To make such requests, please comment to this post (rather than e-mailing us).