PyPy under Cygwin

PyPy (http://www.pypy.org/) provides a Python 2.7 implementation with a Just-In-Time compiler. In order to compile under Cygwin several hurdles have to be overcome:

1. PyPy provides no configuration for Cygwin. I copied the Linux configuration and modified it as needed. In particular I removed the -pthread flag, and the --export-dynamic C linker flag should be --export-all-symbols. This is also where the file extensions (.exe, .dll) are specified.
2. Cygwin does not implement the tm_gmtoff and tm_zone fields in the tm structure, which are a GNU extension to the POSIX standard. PyPy uses these fields to determine the time zone. I implemented workarounds. Use of these tm fields permeates several files that all had to be changed.
3. Under Cygwin the system call waitpid() expects a different argument type for slot 2 as what PyPy uses. I implemented a wrapper that remaps the argument.
4. Under Cygwin the curses C header files are under ncurses.

With these changes PyPy compiles successfully under Cygwin using its current (as of 2012-04-29) python version 2.6.7 and gcc version 4.5.3, under 64-bit Windows 7, 32-bit Windows XP Professional, and 64-bit Windows Server 2003.

Update

I have worked with the PyPy developer community and they have merged most of my patches into the main trunk, so hopefully the adaption of PyPy under Cygwin will get much easier in the future.

Missing libpython2.7.dll issue

While pypy runs just fine for simple programs, it likely will complain about a missing libpython2.7.dll when importing modules. Follow the instructions in ../python/index.html and compile Python2.7.2, or just download the dll from that page. Then install libpython2.7.dll into /usr/bin, and make sure you chmod the permissions to 775.

Memory Mapping Errors

If you get an error saying something about fixup_mmaps_after_fork then this likely has to do with Cygwin's memory architecture (having to map POSIX onto Windows). That is a common issue, and likely related to the Python library libpython2.7.dll. Sometimes this can be fixed by rebasing the system, see the Cygwin Wiki entry at http://cygwin.wikia.com/wiki/Rebaseall. If that doesn't fix it, sorry, you are on your own.

If you get specific errors related to os.fork() then this has to do with the temporary memory the JIT allocates. This has been fixed with the patches for PyPy 1.9 below.


Instructions for PyPy 1.8

A word of warning

This is here only for historic reasons. Any os.fork() calls will likely fail.

Compiling PyPy under Cygwin

1. Get the original sources from this server:
pypy-pypy-2346207d9946.tar.bz2
Unpack with
cd /usr/local
tar -jxvf path/to/your/pypy-pypy-2346207d9946.tar.bz2

2. Apply the patch pypy-pypy-2346207d9946.patch with
patch -p0 < pypy-pypy-2346207d9946.patch

3. Compile. This will take several hours on a fast machine (4 hours on an Intel i7 M620 2.67GHz, 4.5 hours on a 2.8GHz AMD Phenom II X4 830, 6 hours on an Intel Xeon 5130 2GHz).
cd pypy-pypy-2346207d9946/pypy/translator/goal
python translate.py -Ojit

Now you have two options. Either keep the complete build tree, or make a release, and install that.

Make a release and install
4. Make a release.
cd /usr/local/pypy-pypy-2346207d9946/pypy/tool/release
python package.py ../../.. pypy-pypy-2346207d9946-cygwin
This will end up putting an archive into the temp directory. The last line of the output will say where this is, something like: Ready in /c/Temp/usession-release-1.8-7/build

5. Install the newly created release.
cd /usr/local
tar -jxvf wherever/it/said/in/step/4/pypy-pypy-2346207d9946-cygwin.tar.bz2

6. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-2346207d9946-cygwin/bin/pypy pypy

7. Optionally remove the build tree and the temporary files $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Keep build tree and install
4. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-2346207d9946-cygwin/pypy/translator/goal/pypy-c.exe pypy

5. Optionally remove the temporary files. They are $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Installing pre-compiled binaries

Alternatively, download and install the pre-compiled binary distribution.

1. Unpack the pre-compiled binaries pypy-pypy-2346207d9946-cygwin.tar.bz2.
cd /usr/local
tar -jxvf path/to/your/pypy-pypy-2346207d9946-cygwin.tar.bz2

2. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-2346207d9946-cygwin/bin/pypy pypy


Instructions for PyPy 1.9

Compiling PyPy under Cygwin

1. Get the original sources. The latest stable release is available at http://www.pypy.org/download.html#building-from-source. Or get a local copy from this server:
pypy-pypy-341e1e3821ff.tar.bz2
Unpack with
cd /usr/local
tar -jxvf path/to/your/pypy-pypy-341e1e3821ff.tar.bz2

2. Apply the patch pypy-pypy-341e1e3821ff.patch with
patch -p0 < pypy-pypy-341e1e3821ff.patch

3. Compile. This will take several hours on a fast machine (4 hours on an Intel i7 M620 2.67GHz, 4.5 hours on a 2.8GHz AMD Phenom II X4 830, 6 hours on an Intel Xeon 5130 2GHz).
cd pypy-pypy-341e1e3821ff/pypy/translator/goal
python translate.py -Ojit

Now you have two options. Either keep the complete build tree, or make a release, and install that.

Make a release and install
4. Make a release.
cd /usr/local/pypy-pypy-341e1e3821ff/pypy/tool/release
python package.py ../../.. pypy-pypy-341e1e3821ff-cygwin
This will end up putting an archive into the temp directory. The last line of the output will say where this is, something like: Ready in /c/Temp/usession-release-1.8-7/build

5. Install the newly created release.
cd /usr/local
tar -jxvf wherever/it/said/in/step/4/pypy-pypy-341e1e3821ff-cygwin.tar.bz2

6. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-341e1e3821ff-cygwin/bin/pypy pypy

7. Optionally remove the build tree and the temporary files $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Keep build tree and install
4. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-341e1e3821ff-cygwin/pypy/translator/goal/pypy-c.exe pypy

5. Optionally remove the temporary files. They are $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Installing pre-compiled binaries

Alternatively, download and install the pre-compiled binary distribution.

1. Unpack the pre-compiled binaries pypy-pypy-341e1e3821ff-cygwin.tar.bz2.
cd /usr/local
tar -jxvf path/to/your/pypy-pypy-341e1e3821ff-cygwin.tar.bz2

2. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-341e1e3821ff-cygwin/bin/pypy pypy


Instructions for the latest development version of PyPy (currently 1.9.1)

A word of warning

This is current as of mid June 2012. The PyPy sources are under active development, and it is very likely that the patch here will not cleanly work, or not work at all, on future versions of PyPy. If so, you are on your own. On the other hand, going back to the specific revision listed here should always work.

Compiling PyPy under Cygwin

1. Get the original sources. This needs Mercurial installed (get easy_install for python, then run "easy_install mercurial", this will install the command "hg"). The latest build information can be gotten from: http://buildbot.pypy.org/nightly/trunk/. The build file names are something like pypy-c-jit-55492-ac392fb76904-linux.tar.bz2, of this ac392fb76904 is the revision number. The two right columns are color-coded test results, get the revision number of a pypy-c-jit build that is green. For these instructions I assume you are working with revision ac392fb76904.
Get a local copy of the complete development code repository (300+MB).
hg clone https://bitbucket.org/pypy/pypy
Approach 1: Set the working copy to the desired revision.
cd pypy
hg update -r ac392fb76904
Approach 2: Use the latest version from the repository, then there is no need to set the working copy to any desired revision. Or you could set it to a revision that was created after revision c7dff5469611 from June 13, 2012, which already contains the patches from Step 2 below.
cd pypy
Optional: hg update -r [latest and greatest]
2. Only if you chose Approach 1 in Step 1: Apply the patch pypy-pypy-ac392fb76904.patch with
patch -p1 < pypy-pypy-ac392fb76904.patch

3. Compile. This will take several hours on a fast machine (4 hours on an Intel i7 M620 2.67GHz, 4.5 hours on a 2.8GHz AMD Phenom II X4 830, 6 hours on an Intel Xeon 5130 2GHz).
cd pypy/translator/goal
python translate.py -Ojit

Now you have two options. Either keep the complete build tree, or make a release, and install that.

Make a release and install
4. Make a release.
cd ../../tool/release
python package.py ../../.. pypy-pypy-ac392fb76904-cygwin
This will end up putting an archive into the temp directory. The last line of the output will say where this is, something like: Ready in /c/Temp/usession-release-1.8-7/build

5. Install the newly created release.
cd /usr/local
tar -jxvf wherever/it/said/in/step/4/pypy-pypy-ac392fb76904-cygwin.tar.bz2

6. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-ac392fb76904-cygwin/bin/pypy pypy

7. Optionally remove the build tree and the temporary files $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Keep build tree and install
4. Link the executable from a location on your path.
cd /usr/local/bin
ln -s wherever/you/put/it/in/step/1/pypy/pypy/translator/goal/pypy-c.exe pypy

5. Optionally remove the temporary files. They are $TMP/usession-release-1.8* and $TMP/ctypes_configure-* .

Installing pre-compiled binaries

Alternatively, download and install the pre-compiled binary distribution.

1. Unpack the pre-compiled binaries pypy-pypy-ac392fb76904-cygwin.tar.bz2.
cd /usr/local
tar -jxvf path/to/your/pypy-pypy-ac392fb76904-cygwin.tar.bz2

2. Link the executable from a location on your path.
cd /usr/local/bin
ln -s ../pypy-pypy-ac392fb76904-cygwin/bin/pypy pypy


Send comments to mayer@tux.org.

[leftarrow] Back to the previous page.

First posted: Sun Apr 29 17:33:15 PDT 2012
Last updated: Wed Jun 13 11:42:53 PDT 2012