Sample Implementation, Application Battery and LSB Build Status

Week ending May 7, 2004

Jump to: Sample Implementation | Application Battery | Build Tools

LSB Sample Implementation

TODO elements are required for release.

1. Platform build status 7 May: i386, ia64, s390, ppc32, x86_64, ppc64 built to completion. Tarballs of this build uploaded (as 1.91) to snapshots/impl. TODO: s390x segv's building glibc in phase3.
2. Test suite results on ia64 and one of two x86_64 platforms are very poor, with 100+ tests not running due to segfaults in the test binaries. May be library build problems; no chance to investigate. Test journals from running 2.0.2 test suites are available for five of the six built platforms.
3. The static bash which is used in phase2 until a dynamic bash is built may fail after glibc is built. Apparently, even a static bash picks up libnss if it exists, so once the phase2 glibc has been built there may be a problem So far this has only been observed on ia64. Current hack is to build bash twice in phase2, once right after glibc, then later when all the dependencies for the final version are met. That should be removed once bash is really fixed. The hack works on a Debian ia64 host, but fails on a different ia64 host. A fix off the net (for glibc 2.3.1) failed to help. Probably need to build static glibc in bootstrap to make sure libc versions are in sync.
4. ppc64 gcc does not have multilib support, leading to some strangeness in the xml: no lib64 is made, so libgcc_s and libstdc++.so have to be moved in phase2 (phase3 already copies the files manually so is not a problem). Can't configure with --libdir=/usr/lib64 since that also moves /usr/lib/gcc-lib.
5. ppc64 requires a large (250k) patch to binutils, 2.15 would be preferred but is not yet released.
6. With the 2.3.2 glibc, there are a lot of test failures on 64-bit platforms, and it looks like nptl will never work on this setup. We're looking at updating to a new glibc, but tarball-type releases are no longer being done, and the glibc cvs tree doesn't have any new tags either.
7. Have exprimented with unbundled X libs from freedesktop.org, these let us limit what to build to the exact set LSB needs. All of the X libraries have built; Mesa (for openGL) links correctly on ia32 but not on x86_64, appears to be a lib64 issue. This is not being pursued right now as the XFree version seems to be working.
8. TODO: Work remains on file and library cleanup, esp. in coreutils (the existing file cleanup was not carried forward from the old fileutils+textutils+sh-utils that coreutils replaces). A pass was done to identify "extra" shared libraries and binaries with a plan to remove unneeded ones from the lsbsi. The installed files for each package have been captured (there's an nALFS option for this).
9. TODO: Remaining /lib64 issues: a few packages are installing into /lib or /usr/lib where they should not. Several such packages have already been fixed. These are not currently blocking the build but still need cleanup.
10. TODO: The toplevel directory of the lsbsi now reflects the architecture to make it easier for two lsbsi's to be resident on a dual-architecture machine. Examine documentation for impacts.
11. TODO: Test runs have been done on some lsbsi versions. Using 2.0 beta, a lot of unresolveds on an ia64 Debian-host machine - segvs as well as a hang on the new sigwait test. An ia32 (Debian host) run also done, results appear similar but need investigation.
12. TODO: The bootstrap and phase3 need to be verified for the correct build logic. Both install to an alternate root, but when actually running, these are using the standard root. Thus, they should not configure for a non-standard path; rather they should configure a standard path and then use DESTDIR or equivalent when installing. Some time has been spent on this and many of the questionable packages have been fixed, but should be reviewed again.
13. TODO: The lsbsi is still missing install_initd and remove_initd (it provides empty dummies). These are really failures to conform to the spec, but as there's no test they aren't flagged as such by the testing procedure. Still looking for volunteers to work on both a sample implementation and a test suite. An idea raised in the past might be worth considering: the sample could perhaps work on a non-standard location so that "cheaters" who try to install a startup script directly instead of using the required tool to install them could get detected by installing in the lsbsi.
14. TODO: The 'lsb' packages for all architectures need to be released for lsb 2.0. Note this package is now known as lsbsi-lsb, to emphasize it's only intended for the lsbsi. It's currently tagged 1.9, and this is what the lsbsi profile is expecting. This package supplies lsb_release, install_initd, remove_initd, and the proper dependencies. A complication is that because of profiles, and because of multi-arch, the answers a system can give may be not only complex, but may change over time. The simplest way to support that seems to be to have an information directory from which capabilities can be added/removed (/etc/lsb-release.d); the tool will interpret what's there.
15. The UML package needs to be revisited to reflect code changes and newer UML upstream.
16. Bryan Dumm did some work on creating an lsbsi-devel package. The thought is to restore those development tools used by phase 2 but which are not built for phase 3. This includes rebuilding glibc since the development pieces of glibc are removed from phase 3. There are some thoughts on redesigning the build procedure to make this easier.

LSB-si Packages with upgraded base

LSB-si Standards Compliance

This information has been moved.

1. LSB 1.3
2. LSB 2.0

LSB Application Battery

TODO elements are required for release.

1. Appbat Build Status 7 May: headers have been fixed since last week's breakage; everything builds again except Celestia s390: one name mangle problem.
2. All package uplifts planned for LSB 2.0 certification are complete. Some interesting developments include that no patch is needed for Apache any longer; the Samba patch is evolving to where it will only patch the configure scripts, not source files. Both of these indicate we're much closer to upstream buildability. A table of packages with the previous released appbat versions and current upstream versions is included below.
3. TODO: Renaming issue: appbat packages need to move somewhere and be renamed to be FHS-clean. It's not appropriate for us to call a package lsb-apache or put it in /opt/lsb-apache - that might imply it's Apache's officially supported build of Apache for LSB. Location for all appbat programs could be /opt/lsb-appbat or /opt/lsb/appbat (the former would need to be reserved with LANANA to be "legal").
4. TODO: Make the FVT capture-and-submit process more automated where feasible. Currently, you run all the FVTs manually and then assert in your certification application that you did so, and they passed. No results are submitted, and there's no automation. Minimal automation concept is to, where possible, have setup instructions, followed by one or more steps (possibly with further setup in between) that kick off a test and return a pass-fail result, so there's less visual evaluation to determine whether it worked or not. For some apps it's not feasible without a sophisticated test driver tool (xpaint, Celestia, etc.) - basically doesn't work if graphics are involved.
5. TODO: Need an appbat package to use install_initd and remove_initd to install/remove startup scripts as an example.
6. TODO: Check all existing packages for FHS conformance. Possible areas of concern: if an X app-defaults file is installed, does the app find it? Are there /var/run scripts that are being put in /var/opt/pkgname/run instead? etc.
7. Appbat is generally on a strategy of trying to reduce the patches/xml edits to a clearly defined set that would be acceptable as an upstream patch, and at least for packages where it makes sense, pushing them back up. We could get officialy LSB ports out of those projects, or perhaps contribute LSB builds to them.
8. Experiments continue with postgresql. Built successfully using 1.3 tools and 2.0 snapshots. Fixes were committed to 2.0 headers for all issues; minor issues need to be fixed in posgresql for a 1.3 build. Interesting issue raised: how can a package determine its' build target and supply workarounds for buggy LSB headers?
9. Experiments were also done with wine; identified some issues which were fixed in latest wine snapshot; some ioctl issues remain which would need to go into LSB or else wine could be a "requires LSB" build. Wine has supplied us with a list of ioctls they use, with descriptions. We were approached about adding wine to the appbat. This would be i386-only, but it might be interesting to show problems in X libraries, and possibly in program execution areas. Revisit when build is clean.
10. The library battery of static libraries continues to be used for 2.0 building, to get around various problems (such as vendors not consistently making static archives of these libraries available) . Current contents are freetype, freeglut, openssl (libssl and libcrypto), jpeg, tiff, png. A library battery is being built to get around certain build issues, usually vendors not shipping certain static libraries. NEW: libbat may need to be built PIC and add some extra libs, we've uncovered an issue where non-PIC static libraries cannot contribute to a shared library - bad on all architectures, hard failure on x86_64. As the application program ramps up, a library battery might also be needed for LSB 1.3; in general, glibc-2.3-based systems can't fully build the appbat, and most of the 1.3 certifications have been on such systems.
11. libtool has been an ongoing problem. libtool runs before lsbcc/lsbc++ called, so it may find a library file (xxx.la) in the wrong place, and use an explicit path to that library. Thus, having the libbat only helps sometimes. There's a way to call libtool so it looks the right places, still exploring whether it's enough, if dummy .la files are added to /opt/lsb/lib which indicate no shared lib.
12. xpdf will not be upgraded: version 2.x requires Motif for the viewer, resolution of that would have been non-trivial and does not currently advance the state of the LSB so will be left alone.
13. See library dependency graphs at http://www.linuxbase.org/~mats. The dependency tool is in the LSB-futures web tree http://www.linuxbase.org/futures/identification/depends/lsbdepgraph.py.
14. An uplift to latest nALFS was considered, but will not be done now. Current nALFS pre-1.0 xml is not valid valid according to external validators and this restricts the tools available to work with it (e.g. the check/download tool can't use an xml parser to read the package information). The use of nALFS at all remains problematic when appbat is considered a build example to others, as they're unlikely to adopt yet-another-tool. Perhaps should return to rpm-based building, if we could reliably capture all the information to eliminate the reproducibility problems on ia32 that were a problem before. Revisit if we get some more resources to help. There's also the possibility of a tool which would dump the xml in a command-line type format.
15. Old, may need revisiting: some problems detected with xpaint on one platform. Question: does this happen with a native xpaint installed, or not - not sure if the app-defaults for xpaint are going to the right place (i.e., may pick up from native xpaint if one is installed, and not find one if the LSB version puts it in the wrong place and there's no native version).
16. Build Dependency tool - currently nALFS doesn't have a good way to represent build dependencies, may need to build a tool to help with this, esp. when appbat is used outside LSB project as an example. Appbat gets around this through its' configure script.
17. Add new representative sample applications, particularly C++. Candidates: openjade (already built, but not included in the core set)? ImageMagick (has a C++ binding Magick++)? mysql? openssh? ghostscript? lyx? Note: mysql built, openjade already added to appbat (but not in certification bundle). Added to investigation list during the 11Nov call: Mozilla Firebird, the Gimp, bochs, alpha player, flightgear, openvrml, vncserver. Most are C++; some are to test if the build infrastructure has improved enough (e.g. Mozilla was too hard to build in the past, is it better now? the Gimp is not C++ but does use the bottom layer - only - of Gnome: glib,gtk,gdk,gmodule).

Latest Status for Building of Application Battery

Latest Status for Building and Executing LSB Development Environment Checker

Appbat Packages with upgraded base

LSB Build/Check Tools

1. LSB dynamic checker is evolving, will it be ready for LSB 2.0? Could add to certification later.
2. Appchk error reports filed on on sections that don't appear to have variations across implementations on the same architecture - do we have these sections described correctly in the LSB?
3. Autobuilder is running for seven architectures building "1.9.0" snapshots. lsb-build-c++ is not autobuilt; it requires lsbc++ to compile and is done by hand. Another quirk is that libchk now requires a linked binary, which it uses to detect the library locations it's going to test. This currently links against lsb stub libraries, which thus must be installed. A glibc 2.3.2 build host is required.
4. TODO: work out what to do with lsb-build-c++. Still need C++ headers in lsb build env., and won't add this to the DB. Which gcc version to extract from (current 3.2.3)? Do we need anything but headers? Is there a way to produce the heaaders without building the library? Does this still need to be a separate package?
5. Still waiting answers on the "libstdc++ plus LSB patch" proposal. A patch for 3.3 is partly accepted by upstream (for 3.4), but not all. A bigger patch for 3.2 is conceivable. 3.2 and 3.3 are rumored not to be completely compatible, even if 3.3 is built in 3.2-compatibility mode. Further, 3.4 changes the ABI (including the soversion). May need a migration strategy.
6. Looking for a way to support multiple concurrent lsb-build versions. This would help with transition to 2.0; also needed for bi-arch systems which may need to have two build environments. E.g., it's no good if both a 32-bit and 64-bit set of stub libs would install to exactly the same path. The desicion is not to implement this for 2.0.
7. Request: each package should be buildable for /usr/local (default), or other prefix if supplied. LSB build would use something like prefix=/opt/lsb. This mainly works for the four autobuilt test packages through INSTALL_ROOT; it does not work for all of the lsb-build packages which may have some paths built into them today (e.g. lsbcc has hard-coded paths, not a path set as a build-time target). Our packages should be relocatable if possible.
8. Curses issues: we supply curses.h and libncurses.so. Not having two names for both fools some software, should add libcurses.so as a symlink and ncurses.h as a symlink in lsb-build-base.
9. Decide what to do with lsbdev chroot bugs (Chris to look at these when he gets a chance). The chroot is untested since the package renaming/moving.
10. Inactive: investigation of lsbdev tools with a non-GCC compiler. Mats had some luck with this on ia32/ia64, but not without changes. Each compiler is likely to have slightly different build rules for the link step; lsbcc must know about these because by the time the compiler has inserted libs and other rules lsbcc is already out of the picture. E.g., for Intel's compiler, a "compatibility flag" -cxxlib-gcc is needed which lsbcc needs to know about or it will read it as -c; the compiler also requires some extra static libraries which lsbcc needs to know about too. To avoid exposing this to an existing build setup, it was necessary to hack lsbcc; long-term the compilers should support LSB-compatible building themselves. Mats was able to build appbat + extras + libbat, except for Celestia, with icc (8.0 beta) and a modified lsbcc (this was done when only openssl was in the libbat). One way to mitigate the hack-lsbcc problem, if the compiler doesn't support lsb building itself, would be to abstract the linker rules into a config file so different config files could be produced for other compilers.
11. lsb-rpm is not a maintained tool, it was a special build for the chroot to avoid importing extra libs. We would like to have an lsb package build tool, however - one that knew about lsb conformance rules and worked the same everywhere. Best would be if a maintained toool could have an lsb mode, instead of needing to maintain a separate tool or our own patchset. Can we make this happen for rpm (the most likely candidate)? Debian builds an lsb-rpm.
12. pkgchk needs to move towards a release. One round of test/fix was done recently. This is now considered a 2.0 deliverable.
13. Need resolution of lsb dependencies for multiple architectures (e.g. lsb-ppc32 and lsb-ppc64 on a ppc64 machine). Also for the lsb-release package, which provides the same function for non-rpm packages (affects release of lsbsi-lsb pkg) - need to define the syntax for this. In addition, a request was made to define, at the same time, how this will be extended for profiles (without necessarily defining the profile names yet - just define the naming scheme). See http://www.linuxbase.org/~mats/multi_arch.html for more discussion.