Saturday, May 14, 2011

Judges' machine and system environment

As stated in the contest announcement, submitted programs must be run by the judges.  We plan to use the following environment:
  • We will run the submitted programs on the system-level (as opposed to language-level or application-level) virtual machine KVM with QEMU.
  • The host operating system will be Debian GNU/Linux 5.0 (lenny) for amd64.  The guest will be Debian GNU/Linux 6.0 (squeeze) for amd64.  The host OS is older than the guest for administrative reasons.  For the same reasons, we plan to use the version of KVM/QEMU in the "kvm" (not "qemu") package of lenny.
  • The host (physical) machines will be a cluster of Intel Xeon E5530 2.4 GHz CPUs (Intel VT enabled) with 24 GB main memory each (or another cluster with similar hardware, depending on availability).  Each guest (virtual) machine will have a single-core CPU provided as the default of KVM/QEMU, 1 GB main memory with 2 GB swap and 30 GB (virtual) disk space.
  • Each submitted program will be run by a unique general user (as opposed to a shared or privileged account) on the guest OS.  We plan to impose the hard limits of 512 MB virtual memory and 1 GB disk usage on each submission.  [Update on May 27: For technical reasons, we will actually impose the hard limits of 8 MB stack (the default of Debian squeeze) and 512 MB data segment, and no limit for virtual memory (address space used), on each process.  However, any program whose total memory footprint (the sum of the resident set sizes of all of your processes running simultaneously) exceeds 512 MB at runtime (as opposed to installation time) may be disqualified by the judges.  Usually, one can check it by watching the "RES" field of the "top" command.]  There will also be a hard limit for CPU usage, which will be announced in the task description when the contest starts.  No network connection will be available for the submitted programs at runtime.
  • We accept requests for installing standard Debian squeeze packages (those listed in http://packages.debian.org/squeeze/) required for running (not compiling) your program, from now on until the contest endsPlease specify the exact name(s) of the package(s) in a comment to this post (do not e-mail to the organizers or comment to other posts for this purpose).  Please also check the list of already installed packages to make sure that your request is not subsumed yet.
  • If the runtime/libraries/files required for running your program is not available as a standard package, submit them together with your program.  We will install your submission in the home directory of a unique general user.  [Update on May 15: If your program needs to know the absolute path of this directory, please try to use $HOME or something similar.  In case this is impossible, e-mail to icfpc2011-blogger AT kb.ecei.tohoku.ac.jp and request a specific username (please try to choose a unique one).]  Details of the submission procedure will be described in the contest task.
  • In either case, we will do our best to run each submitted program.  For this purpose, we may contact you (by e-mail to the address registered at the time of submission) after the contest has ended.
  • Source code of your program must also be submitted, though it will not be compiled by the judges (except for the winners, whose submissions will be examined with more attention).
  • We will not distribute the image of our virtual machine because of network bandwidth limitations.  It should anyway be easy for oneself to install Debian squeeze on a virtual (or physical) machine.
Please note that all the above is subject to possible changes in case of unexpected problems of the hardware and/or software.  We appreciate your understanding.  Thank you!

[The ICFP Programming Contest 2011 organizers, with special thanks to the InTrigger team]

[Update on June 13: It would be the best if each participant can prepare an environment (virtual or physical) with Debian squeeze for amd64 (which supports x86-64 including Intel 64), but in case you can only produce 32-bit executables, please remember:
  • Statically linked 32-bit Linux executables should run with no problem on the 64-bit environment;
  • Dynamically linked 32-bit libraries should also be fine as long as they are included in the ia32-libs package of Debian; and
  • Other dynamically linked 32-bit libraries (and any other files required at runtime but unavailable on Debian squeeze for amd64) can be included in the submission itself.]
[Update on June 14: Most virtual machine software for Intel PC requires that you enable Intel VT in your host BIOS to run 64-bit guest OS.  Please see the documents of your VM (and PC) for details.]

98 comments:

  1. I would respectfully like to register my disappointment in the fact that the judge's test system will only have access to a single CPU core. The ability to safely and efficiently construct concurrent solutions has been a shining light of functional programming. To hobble this ability with the use of only a single core seem to me to be against the spirit of the IFPC Programming Contest's aims. I ask that the judges reconsider this limitation.

    ReplyDelete
  2. Unfortunately, it is a limitation of the system we have access to. Thank you for your understanding.

    ReplyDelete
  3. Not knowing the nature of the task (interpreted languages may be totally inappropriate), but can you add package "ruby1.9.1", please?



    Team taxidriver

    ReplyDelete
  4. Request granted and list updated:
    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  5. re vm image distribution: wouldn't it be possible to set up a torrent (as a way to limit bandwidth used)?

    ReplyDelete
  6. Unfortunately, many universities restrict P2P communication nowadays, including ours. It is anyway very easy (in our experience, even easier than downloading a VM image!) to install Debian squeeze on VirtualBox, VMware, etc.

    ReplyDelete
  7. >> We plan to impose the hard limits of 512 MB virtual memory

    Does this mean the virtual machine will only have 512 MB of ram or that our process will be monitored and killed if it goes over 512 MB allocated?

    ReplyDelete
  8. Will we be able to know the full pathname of the directory where our program is installed, so that we can easily hard-code that path to the executables we build?

    I presume this could be implemented easily by each team automatically getting a unique short alphanumeric id when they send their first sumbission, and the directory being a function of that id.

    ReplyDelete
  9. Please try to use $HOME or something similar. In case this is impossible, e-mail to icfpc2011-blogger AT kb.ecei.tohoku.ac.jp and request a specific username (please try to choose a unique one).

    ReplyDelete
  10. > Does this mean the virtual machine will only have 512 MB of ram or that our process will be monitored and killed if it goes over 512 MB allocated?

    The latter. As specified in the original post, the virtual machine has more memory.

    ReplyDelete
  11. Anonymous CowardMay 17, 2011 at 2:40 AM

    Will the test machines have ia32-libs on it and be able to run binaries compiled for 32 bits?

    ReplyDelete
  12. Please provide the llvm and llvm-dev packages.

    Thanks,
    The higher order of zeuxis

    ReplyDelete
  13. No Erlang packages available?

    ReplyDelete
  14. As requested, packages "ia32-libs", "llvm", "llvm-dev", and "erlang" have been installed:

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    As stated in the original post, please specify the exact names of the Debian squeeze packages you request (rather than asking "is XXX installed?").

    ReplyDelete
  15. Please install Python and Psyco on the machine.

    ReplyDelete
  16. > Please install Python and Psyco on the machine.

    As stated in the original post (and the previous comment), please specify the exact names of the Debian squeeze packages:

    http://packages.debian.org/squeeze/

    and please also check the list of already installed packages:

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  17. Please add:

    scala (2.7.7.dfsg-4)
    scala-library (2.7.7.dfsg-4)

    Thanks!

    ReplyDelete
  18. Packages "scala" and "scala-library" are now installed:

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  19. Packages requested:

    mlton
    smlnj
    libsmlnj-smlnj

    liblua5.1-0
    liblua5.1-0-dbg
    liblua5.1-0-dev
    liblua5.1-copas-dev
    liblua5.1-copas0
    liblua5.1-doc0
    liblua5.1-expat-dev
    liblua5.1-expat0
    liblua5.1-filesystem-dev
    liblua5.1-filesystem0
    liblua5.1-json
    liblua5.1-logging
    liblua5.1-logging-dev
    liblua5.1-lpeg-dev
    liblua5.1-lpeg2
    liblua5.1-md5-0
    liblua5.1-md5-dev
    liblua5.1-posix-dev
    liblua5.1-posix1
    liblua5.1-socket-dev
    liblua5.1-socket2
    liblua5.1-sql-sqlite2
    lua5.1

    ReplyDelete
  20. Installed, except for two unfound packages "smlnj" and "libsmlnj-smlnj". I am afraid they are not included in the standard distribution of Debian squeeze. Please include the necessary runtime in your own submission.

    ReplyDelete
  21. Is possible to get libmono-system-runtime2.0-cil ?

    ReplyDelete
  22. Sure, of course!

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  23. pari-gp
    pari-extra
    pari-gp2c

    Thanks!

    ReplyDelete
  24. How about gnu-smalltalk and gforth ?

    ReplyDelete
  25. Installed.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  26. Please install
    sun-java6-bin (6.24-1~squeeze1) [non-free]
    sun-java6-jdk (6.24-1~squeeze1) [non-free]
    sun-java6-jre (6.24-1~squeeze1) [non-free]

    and remove
    openjdk-6-jre
    openjdk-6-jre-headless
    openjdk-6-jre-lib
    gcj-4.4-base
    gcj-4.4-jre
    gcj-4.4-jre-headless
    gcj-4.4-jre-lib
    gcj-jre
    gcj-jre-headless

    OpenJDK is not portable enough and not well-tested (compared to Sun's reference implementation). Thus people writing in Java/JVM-based languages wouldn't be able to develop on a different platform and be sure that it actually runs correctly on OpenJDK.

    GCJ is know to break a lot of things in Java and it shouldn't be used. When GCJ is installed it's often tries to be "the" java, making conflicts for the software which tries to use the real Java. Please remove it (and all the packages that depend on GCJ specifically, like openoffice.org-gcj, bash-gcj and some libraries that have gcj in them).

    ReplyDelete
  27. Please install 'sun-java6-jdk' and its dependencies.

    ReplyDelete
  28. Granted.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  29. Sure.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  30. Please install python-psyco (http://packages.debian.org/en/squeeze/python/python-psyco)

    ReplyDelete
  31. >> our process will be monitored and killed if it goes over 512 MB allocated?

    > The latter

    Can you please review this? Many runtimes used to overcommit memory. For example, right now I ran nearly empty java process and VSZ goes as high as 650M. And sometimes it is very hard to change.

    ReplyDelete
  32. We have checked that Java (Sun JRE 1.6.0_24 with the default options) runs under the 512MB virtual memory use hard limit on judges' environment. If there is a real problem with this limit, please let us know again.

    ReplyDelete
  33. It was:
    openjdk-6-jre 6b18-1.8.3-2+squeeze1
    clojure 1.1.0+dfsg-1

    $ clojure
    VSZ=679708
    I did not try sun jre

    then,
    sbcl 1:1.0.47.0-2
    $ sbcl
    VSZ=538156

    That all was debian 32bit.

    ReplyDelete
  34. Please install:

    ocaml-nox
    ocaml-findlib
    camlp4-extra
    ocaml-native-compilers
    libextlib-ocaml-dev
    libcore-extended-ocaml-dev
    libcore-ocaml-dev
    liblwt-ocaml-dev
    libbatteries-ocaml-dev
    libocamlgraph-ocaml-dev
    libpcre-ocaml-dev
    libbitstring-ocaml-dev

    Thx

    ReplyDelete
  35. You say that we need to submit the compiled code and the source code is not enough.
    What if I submit the source code (say C++) and a script (say bash) compiling and then running the code?

    ReplyDelete
  36. please add haskell-platform

    ReplyDelete
  37. Thank you, max630. We will consider what we can do, and will respond to your request as soon as possible.

    ReplyDelete
  38. I am afraid that "python-psyco" is not available as a standard Debian squeeze package for amd64. If necessary, please consider including its runtime (if any) in your submission.

    ReplyDelete
  39. > What if I submit the source code (say C++) and a script (say bash) compiling and then running the code?

    That is OK, but there would be a risk that the compilation script fails for some reason. Please avoid that as far as possible.

    ReplyDelete
  40. Requested packages (related to OCaml and Haskell) have been installed:

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  41. In response to previous comments (by max630) regarding the hard limit on virtual memory use, we have updated our specification of the environment. Please see "Update on May 27".

    ReplyDelete
  42. Python 3 and PLT Scheme, please :)

    python3
    plt-scheme

    http://packages.debian.org/squeeze/python/python3
    http://packages.debian.org/squeeze/lisp/plt-scheme

    ReplyDelete
  43. Hello,
    please install the package

    groovy

    Best,
    Maxym

    ReplyDelete
  44. please disregard my previous comment.

    Debian's groovy package is unfortunately too old to be usable.

    Sorry, Maxym

    ReplyDelete
  45. > Python 3 and PLT Scheme, please :)

    Acknowledged.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    > Debian's groovy package is unfortunately too old to be usable.

    We are sorry about the limitations of our environment - our manpower is finite and we had to draw the line somewhere.

    ReplyDelete
  46. Why wouldn't you put vm image on some public tracker?
    Do it from home if university torrent policy is so strict.

    ReplyDelete
  47. Do you have a more specific suggestion? The many gigabytes of VM image resides on a university cluster and is updated upon every installation request. (Please also understand that, although we are doing our best, we are volunteers and doing these in our spare time.)

    And once again, installing Debian squeeze on your VM would be _much_ faster and easier than downloading the VM image. Please just try if you doubt.:-) Thank you!

    ReplyDelete
  48. Hello,
    please install the package

    python-gmpy

    Best,
    Sergey

    ReplyDelete
  49. With pleasure.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  50. For what it's worth, I'm using the following very simple script to keep my VirtualBox-base Debian squeeze VM up-to-date (apologies for the bad formatting):

    wget -N 'http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt'
    sudo apt-get install $(
    cat apt-list.txt \
    | sed -e 's!^....\([^ ]*\).*!\1!' \
    | grep -v -E '^(xserver-xorg-video-siliconmotio)$'
    )

    I need to filter out one of the xserver-xorg packages, which cannot be installed for some reason: apt-get gives me "E: Unable to locate package xserver-xorg-video-siliconmotio". (And this is not because of the chopped off package name.) But this is not a package one would require, I imagine. Although, who knows... :-)

    ReplyDelete
  51. ACK.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  52. Hello,

    I've never installed a linux distribution on VMWare. Could you please provide a link explaining how it is done, if possible specific to the distribution you require?

    Thanks,

    Alan

    ReplyDelete
  53. Could you please include Gauche in the environment?

    gauche (0.9-16)
    gauche-c-wrapper (0.6.1-4)
    gauche-dev (0.9-16)
    gauche-gdbm (0.9-16)
    gauche-gl (0.4.4-5)
    gauche-gtk (0.4.1-13)
    gauche-zlib (0.9-16)

    ReplyDelete
  54. > I've never installed a linux distribution on VMWare. Could you please provide a link explaining how it is done, if possible specific to the distribution you require?

    We are sorry but we do not have sufficient manpower to provide technical support for basic software. Please read the manuals and/or search the WWW (or just try and see - most if not all of the installation procedures are self-explanatory).

    ReplyDelete
  55. > Could you please include Gauche in the environment?

    Done.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  56. spidermonkey-bin please.

    ReplyDelete
  57. Got it.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  58. libmath-gmp-perl
    libmath-bigint-gmp-perl
    libanyevent-perl
    libarchive-zip-perl
    libmodule-install-perl
    libdbd-sqlite3-perl
    libdbi-perl
    libev-dev
    libev3
    liblist-moreutils-perl
    libreadonly-xs-perl
    libreadonly-perl

    ReplyDelete
  59. Roger.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  60. Just to clarify, we'll need to submit binaries for amd64? Does this differ from x86-64? I'm afraid I can't find among our team any hardware specifically with an AMD 64-bit CPU in it.

    ReplyDelete
  61. > zsh please

    Here you are.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  62. > Just to clarify, we'll need to submit binaries for amd64? Does this differ from x86-64? I'm afraid I can't find among our team any hardware specifically with an AMD 64-bit CPU in it.

    Any executable is fine as long as it can be run on our guest environment. For more information on Debian and amd64 (and perhaps some virtual machine software), please read the documents and/or search the WWW, for example:

    http://www.debian.org/ports/amd64/index.en.html

    > The port consists of a kernel for all AMD 64bit CPUs with AMD64 extension and all Intel CPUs with EM64T extension, and a common 64bit userspace.

    ReplyDelete
  63. Speaking of amd64, would a binary created in a VM running the i386 version of Squeeze work on your system?

    ReplyDelete
  64. Again, we are sorry but we cannot provide technical support for individual languages. Please prepare a compatible environment (on a virtual machine if necessary) and check by yourself.

    ReplyDelete
  65. P.S. See also (again):

    http://www.debian.org/ports/amd64/index.en.html

    > Native execution of legacy 32bit binaries is supported by the kernel, and core libraries are provided by the ia32-libs package.

    However, this mainly applies to C (and C++) libraries only. As we wrote above, we respectfully ask you to test your language on an environment compatible to ours (again on a virtual machine if necessary) well in advance.

    ReplyDelete
  66. I guess we're not understanding each other: I want to know if installing the i386 version of Squeeze in a VM is a "compatible environment". (I'm installing on VMWare on a MacBook Pro with a Core 2 duo processor.)

    To rephrase: how can I test if my environment is compatible?

    Thanks,
    Alan

    ReplyDelete
  67. > I want to know if installing the i386 version of Squeeze in a VM is a "compatible environment"

    The short general answer is no, of course, because i386 is a different architecture (32-bit) from amd64 (64-bit). In particular, some dynamically linked 32-bit libraries may not be available on a 64-bit environment (though some 32-bit "core libraries" are provided as the "ia32-libs" package in Debian).

    Just in case you are not familiar with the terminology, both AMD64 and Intel 64 are implementations of x86-64 and are compatible with each other (modulo a few minor differences which are unimportant in most operating systems). A Google search for "amd64" gives the following page at the top.

    http://en.wikipedia.org/wiki/X86-64

    > To rephrase: how can I test if my environment is compatible?

    Please just install Debian squeeze for amd64 on your VMware. It is also compatible with Core 2 Duo. Once again, a Google search for "Debian amd64" gives at the top:

    http://www.debian.org/ports/amd64/index.en.html

    I hope these help (and please read the documents and/or search the WWW by yourself before asking us questions about general hardware and software - thank you very much for your understanding and cooperation!).

    ReplyDelete
  68. I had found this last page, which does not mention core 2 duo by the way, and my question was: is the included support for i386 enough, which you just answered. Thanks a lot! And one last clarification: these were not "general" hardware and software questions, they were quite specific ones about the very specific requirements of this contest.

    ReplyDelete
  69. Thanks for the clarifications. It is the best to prepare a Debian squeeze environment for amd64 (virtual or physical), but just in case you can only produce 32-bit executables, please remember:

    - Statically linked 32-bit Linux executables should run with no problem on the 64-bit environment.

    - Dynamically linked 32-bit libraries should also be fine as long as they are included in the ia32-libs package of Debian.

    - Other dynamically linked 32-bit libraries (and any other files required at runtime and not available on Debian squeeze for amd64) can be included in the submission itself.

    I am sorry for my poor English but all of these are direct consequences of readily available information on the Web. We truly appreciate your cooperation. Thank you very much!

    Eijiro

    ReplyDelete
  70. Could you clarify whether OpenJDK and GCJ are removed as requested by Ivan T?

    ReplyDelete
  71. Yes, they have been removed.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  72. Could you please install:

    php5-cli

    Thanks!

    ReplyDelete
  73. Would you please add packages flex and bison?

    ReplyDelete
  74. > php5-cli
    > Would you please add packages flex and bison?

    Processed.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  75. minisat2
    picosat

    Thanks - J.W.

    ReplyDelete
  76. You're welcome.

    http://www.kb.ecei.tohoku.ac.jp/~sumii/tmp/apt-list.txt

    ReplyDelete
  77. Is it possible to get the output of dpkg --get-selections rather than the apt-list.txt file you have? I don't know if it's possible to automate installations from the file you've provided, but from the get-selections output it's trivial to ensure all available packages are installed in my development image: http://lists.samba.org/archive/linux/2002-June/003399.html

    ReplyDelete
  78. Thank you for the suggestion!

    http://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt

    See also:

    http://www.icfpcontest.org/2011/05/judges-machine-and-system-environment.html?showComment=1307110031039#c1196844878145834435

    We will keep both lists up-to-date.

    ReplyDelete
  79. Package request:

    mono-complete

    ReplyDelete
  80. I know this isnt an official debian package, but its possible to have the f# package from http://fsxplat.codeplex.com/
    ?

    thanks anyway.

    ReplyDelete
  81. Will you add

    emacs23

    to the list, so that I (and other emacs users) can edit our code on the VM?
    I guess I can install emacs by myself and the difference between the judge's machine and mine will bring no inconsistency, but just in case.

    ReplyDelete
  82. > I know this isnt an official debian package, but its possible to have the f# package from http://fsxplat.codeplex.com/ ?

    We are sorry but we cannot install non-official packages. Please include in your submission whatever is necessary for _running_ (not _compiling_) your program.

    ReplyDelete
  83. > mono-complete

    > emacs23

    Installed.

    http://www.kb.ecei.tohoku.ac.jp/icfpc/apt-list.txt (we move this file but the previous address will also remain valid)
    http://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt

    You already konw this but just to make sure for others, participants do not have to _develop_ their programs on the VM; it is just for _testing_ their submissions.

    ReplyDelete
  84. Could you please install libneko0 package?

    ReplyDelete
  85. Could you please install "neko" package? I'm not sure whether libneko0 will be enough.

    ReplyDelete
  86. > libneko0
    > libneko

    Installed to the unofficial duel server.
    Will soon be installed to the official machine image, too.

    ReplyDelete
  87. fp-* and fpc-* packages aka FreePascal thingies please. :)

    ReplyDelete
  88. > fp-compiler
    > fp-ide
    > fp-units-base
    > fp-units-db
    > fp-units-fcl
    > fp-units-fv
    > fp-units-gfx
    > fp-units-gnome1
    > fp-units-gtk
    > fp-units-gtk2
    > fp-units-math
    > fp-units-misc
    > fp-units-multimedia
    > fp-units-net
    > fp-units-rtl
    > fp-utils
    > fpc

    are installed to the unofficial server.
    Will also be included on the official machine.

    ReplyDelete
  89. Hi everybody, will you please install python-dev [1] ?

    thanks in advance
    sciamp

    [1] http://packages.debian.org/squeeze/python-dev

    ReplyDelete
  90. Due to a number of requests and technical reasons, we have installed the F# package available from:
    http://fsxplat.codeplex.com/releases/view/48831

    ReplyDelete
  91. Hi, can you please install libboost-program-options-dev?
    Thanks in advance!

    ReplyDelete
  92. > libboost-program-options-dev

    Processed.

    ReplyDelete
  93. By request, we have installed Ruby 1.8 packages.

    http://www.kb.ecei.tohoku.ac.jp/icfpc/apt-list.txt (we moved this file but the previous address will also remain valid)
    http://www.kb.ecei.tohoku.ac.jp/icfpc/dpkg-selections.txt

    To make such requests, please comment to this post (rather than e-mailing us).

    ReplyDelete