For a few years now, my main task at work has been working on the firmware of an IP phone. The phone runs VxWorks on a MIPS32 CPU; the firmware is written in C and C++.

For slightly less time, I've been dabbling in python on my own time. Freedom from explicit typing was a refreshing change, and python's tendency to Just Work was a nice bonus.

It was perhaps inevitable that I would one day try to combine phone and language. (why? because they were both there)

It wasn't obvious that the idea stood a chance. VxWorks is a bit off the OS beaten track, and might not provide all the functionality needed by Python's "core" (not with the same names, anyway). There might be some processor-specific pieces that would rule out MIPS32. And even if I could get something built, would it fit in the 2 or 3 MB of RAM (and even less flash) I could spare?

As it turns out, there was very little to worry about. Python's code is impressively (if perhaps unsurprisingly) portable, only needing a couple of tweaks to its build system and none at all to its source code. There doesn't seem to be anything CPU-dependant; and in the end, adding python to my firmware only cost me 1MB. It took me only a few evenings of tinkering to get a libpython built, linked into my firmware, and loaded on my phone, to the point that I could run this little experiment at the VxWorks shell:

  -> Py_Initialize()
  value = 42 = 0x2a = '*'
  -> PyRun_SimpleString("print 'Hello, World!'\n")
  Hello, World!
  value = 0 = 0x0

(the VxWorks shell being a peculiar animal that allows calling C functions by name, in this case giving me access to Python's C Extension API for a near-REPL experience)

For my purposes, that's enough; I know it can work, and that's all I wanted. I don't expect to ever go further than this. But as little as it is, publishing how I got there might help someone get started on a real project; so here goes:

Porting python in 10 easy* steps

*for a suitable definition of easy

1. As far as embedding Python in an existing application (or firmware) is concerned, Python's own documentation should give you most of what you need
   
2. You'll need a cross-compiling toolchain, i.e. a compiler that can be used on one platform (e.g. x86) and produces executables for use on a different platform (e.g. MIPS32). GCC is your best bet; it's what will make python's build system happiest, and there's lots of resources on getting a GCC cross-compiler working on the web, though it looks a bit daunting to me. I was fortunate in that, since I was already set up to build firmwares, I already had all the needed tools; I would guess that most people engaging on a similar project would be in the same position.
   
   
3. In addition to the compiler (and assembler, linker, etc), you'll want to have a Unix-like environment to run Python's configure script and makefiles. If you're on Windows, cygwin will serve nicely.
   
   
4. The 'configure' script needs some tweaking: it contains a few uses of AC_TRY_RUN, which will fail when cross-compiling.
   
   • If you have a working 'autoconf', the simplest is to edit the 'configure.in' file. You can either remove the AC_TRY_RUN tests altogether or replace them by the newer, more cross-compiler-friendly AC_RUN_IFELSE. Then run 'autoconf' to regenerate the 'configure' script.
     
   • If you don't (as I didn't), you can brace yourself and go edit the 'configure' script directly. Running the script produces error messages that gives something to search for. The fix is actually simple: just remove the calls to 'exit' to allow the error to get ignored.
     
   
   
   
5. The makefile also needs tweaking: just like 'configure', at some point it tries to compile and run a program. This appears to be in order to autogenerate some source files, which fortunately are already provided in the source distribution; so it's safe to disable this step. The simplest way:
   
   • open "Makefile.pre.in"
     
   • find the place where "$(PGEN)" shows up AFTER a ':'
     
   • remove "$(PGEN)"
     
   
   (this will only prevent the executable from getting built. The makefile will still attempt to run it, but it's written so that the resulting failure is ignored)
   
   
6. The configure script and makefile try to guess at the name of tools to use; you can give them a hint with environment variables. In my case I needed to set CC (the C Compiler) and AR (the "archiver", ie. what creates static libraries)
   
   
7. If you need to specify special command-line options to the compiler, environment variables can also be used. Annoyingly, 'configure' and the makefiles use different variable; you'll want to set CFLAGS and BASECFLAGS to the same thing.
   
   
8. Finally you'll be ready to run the 'configure' script. You need to give it the special options --build and --host to tell it you're cross-compiling, something like:
   

   
   $ ./configure --build=win32 --host=vxworks
   

   
   
   (win32 and vxworks were a wild guess that happened to work for me. I got the impression the specific values didn't particularly matter)
   
   
9. You can then run 'make' to compile everything. If, like me, all you need is a static library, this will do it:
   

   
   $ make libpython2.5.a
   

   
   
   
10. There's a good chance some files under Modules/ will fail to compile (in my case, posixmodule.c). The file Module/Setup specifies (in a rather well-documented way) which Python modules (written in C) should be built into the python library; comment out the failing one, and re-run 'make'. I only had to disable posixmodule and pwdmodule; YMMV.
    

And for me, that was it; nothing else needed manual intervention. If you run into more troubles (e.g. trying to build the actual python.exe), I'm afraid you're on your own.

My next step was to figure out how to integrate the python library into my firmware; you'll have to figure out the corresponding steps for your own firmware/embedded application/whatever. Start with the 'embedding' link for how to access python code from your code.

If you want to be able to load python source files with 'import', pay particular attention to what that page says about PYTHONHOME; as for me, I put a putenv(PYHONHOME=/whatever") before the Py_Initialize call, letting me import /whatever/python2.5/*.py files (and possibly, though I haven't tried, .py files contained in a /whatever/python2.5/libpython2.5.zip)

Happy cross-compiling!

Labels: c++, cross-compiling, embedded, python

# posted by LPG @ 4:30 p.m.