Name ==== Patrick "P. J." McDermott Contact/Email ============= mailto:pjm@nac.net mailto:pjm28@njit.edu "pehjota" on Freenode and OFTC IRC networks Background ========== I am a 21-year-old undergraduate computer science student at the New Jersey Institute of Technology (New Jersey, United States) with plans to graduate in May. At the university, I have led a special interest group for GNU/Linux; organized events related to free software; and given workshops and tutorials on GNU/Linux, UNIX [1], Debian, and Git. I use Debian GNU/Linux on my desktop, laptop, and server. My technical and theoretical interests include operating systems and distributions, package management, toolchains, hardware interfaces, embedded systems, networking, and graph theory. Technical Skills ---------------- I am proficient or knowledgeable in numerous programming, markup, and formatting languages, including ISO C 89 and 99, DOT, groff (with the "man" macro set), POSIX make, Markdown, and UNIX shell. [2] I have about six years of experience in programming and over eight years of experience in Web design. Common design goals for my software projects include modularity, portability, and efficiency. I am proficient with the use of Debian GNU/Linux and its tools. I have about one year of experience using Git, and I use it almost daily for software projects, coursework, etc. I have some experience with Debian packaging, however I do not yet maintain any packages in the Debian archive. I am comfortable using mailing lists, IRC, and the Debian BTS. Projects -------- Currently I am leading the development of a new multipurpose embedded operating system distribution. Notable software packages include Linux-libre, BusyBox, GCC, EGLIBC, and opkg (a lightweight package manager and frontend similar to dpkg and APT). For the project I designed a source package format [3], wrote a set of package building tools [4] (similar to dpkg and debhelper), and designed ways to configure and build the distribution for specific hardware and applications. We are now packaging software and building a cross toolchain [5] to port the system to its first target (an ARM Cortex-A8 CPU). In preparation for the port, I've identified circular build dependencies [6] and worked on methods to bootstrap the building (using package status overrides and simplified builds) and installation of packages for an architecture. Previous projects on which I've worked include an implementation of Conway's Game of Life in C using curses [7], a peer-to-peer chat client written in C with a curses interface, and a small C library to read and write run control files [8]. Project title ============= Bootstrappable Debian Project details =============== In Debian there are a number of build dependency cycles – sets of one or more source packages in which the building of each requires at least one binary package built from a member of the same set. Below are three examples of build dependency cycles: * The source package `make-dfsg`, like most if not all packages, cannot be built without a `make` program. However, the `make` program most commonly used to build Debian packages is that provided by the `make` binary package built from `make-dfsg`. * The `pkg-config` utility links against GLib, and so its Debian source package of the same name build-depends on `libglib2.0-dev` (built from `glib2.0`). However, GLib (and therefore the `glib2.0` source package) calls on the `pkg-config` utility to help it compile and link against other libraries. * The source package `krb5` requires `libldap2-dev` (built from `openldap`) to build. Similarly, building `openldap` requires `libkrb5-dev` (built from `krb5`). When building a Debian system from scratch – e.g. to build a new Debian port, to update an older architecture, or to build packages for non-native architectures – these build dependency cycles must be broken. This is commonly done by building some packages in a "staged" form with minimal dependencies, e.g. by disabling links against some libraries or by providing information manually. With the staged binary package(s) built and installed, depending packages can be built and installed. Finally, the staged packages can be built normally. The first part of this project is to analyze build dependency cycles in Debian (by generating directed graphs in DOT language and drawing them using Graphviz), to determine how they can best be broken, and to modify certain packages involved in such cycles to support staged bootstrap builds. New control fields `Build-Depends-StageN` will be added to the packages' `debian/control` files, and the `debian/rules` files will be modified to produce reduced/staged builds of packages depending on the value of the `DEB_BUILD_OPTIONS` environment variable. Staged binary packages should be marked as such and not uploaded to the Debian archive. A marking solution is not yet finalized, but a new control field seems the obvious one. Also useful (especially for humans) could be a version string suffix or a new file extension (like "udeb" and "tdeb"). Individual packages can be tested using patched versions of Debian package management and building tools. Patches for these tools exist and may be used as they are or modified (e.g. to support `Build-Depends-StageN` fields for any value of `N`). To build a complete Debian system from scratch, however, requires cross building at least an initial core set of packages. Many of these packages currently fail to cross build. The second part of this project, therefore, is to identify which packages need to be cross built before the rest of the packages can be natively built and to patch any packages that fail to cross build as necessary. This initial core set of packages to be cross built will probably include at least those with a `Priority: required` field or `build-essential` as a reverse dependency. The "port bootstrap build-ordering tool" project can help in determining which packages should be cross built. Synopsis ======== This is a project to break build dependency cycles in Debian by modifying packages to support staged bootstrap builds. Benefits to Debian ================== Building a Debian system from scratch – e.g. to port Debian to a new architecture – is currently very difficult due to cyclic build dependencies and the need to cross-build at least an initial set of packages. This project would make it easier to bootstrap new Debian ports, update older architectures, and build packages for non-native architectures (especially those for which building packages natively might be unreasonable or impossible). Deliverables ============ * Patches for `debian/control` and `debian/rules` files of packages to support staged bootstrap builds that break build dependency cycles. * Patches for build tools to handle the new control fields and support cross-building of the initial core set of packages. * Patches for an initial core set of packages to support multiarch cross builds. Project schedule ================ * 2012-05-01 - 2012-05-20 * Continue to study documentation related to bootstrapping of Debian and Ubuntu. * Become more familiar with Debian Multiarch (specifically the control field and implementation in packaging and build tools). * Generate directed graphs of build dependency cycles in DOT language, draw them using Graphviz, and analyze them to find packages the staging of which most easily breaks the cycles. * Become more familiar with Debian build and bootstrap tools (e.g. pbuild and debootstrap). * 2012-05-21 - 2012-05-27 * Apply and test patches for Debian build tools to handle new control fields. Attempt to support `Build-Depends-StageN` fields for any value of `N`. * 2012-05-28 - 2012-06-16 * Modify the estimated 40 packages (excluding toolchain packages) necessary to break build dependency cycles. (At least 10 should be patched per week.) * 2012-06-17 - 2012-07-01 * Modify the toolchain packages as necessary to break build dependency cycles. * 2012-07-02 - 2012-07-08 * Build a multiarch toolchain to cross build packages. * 2012-07-09 - 2012-07-13 * Work on and submit mid-term evaluation. * 2012-07-14 - 2012-07-29 * Modify the packages with a `Priority: required` field to support multiarch cross building. * 2012-07-30 - 2012-08-05 * Modify dependencies of `build-essential` to support multiarch cross building. * 2012-08-06 - 2012-08-13 * Modify any other packages found necessary to make a cross-built minimal bootstrappable system that can natively bootstrap build other packages. * 2012-08-14 - 2012-08-19 * Review patches for quality. Exams and other commitments =========================== I have no exams or courses this summer. My spring semester final exams end on 2012-05-08. I graduate on 2012-05-15. Other summer plans ================== I have no other plans (jobs, vacations, or the like) for the summer except to spend some time working on other free software. I can work full-time on this project. Why Debian? =========== I am a proud user of Debian GNU/Linux, and I admire the Debian Project's infrastructure, commitment to quality, and attention to legal issues and user freedom. Applications to other organizations =================================== I am not applying to any other organizations. References ========== 1: 2: 3: 4: 5: 6: 7: 8: