Sample Implementation, Application Battery and LSB Build Status

Week ending March 5, 2004

LSB Build Tools

1. Autobuilder is running for seven architectures building "1.9.0" snapshots. lsb-build-c++ is not autobuilt; it requires lsbc++ to compile and is done by hand. Another quirk is that libchk now requires a linked binary, which it uses to detect the library locations it's going to test. This currently links against lsb stub libraries, which thus must be installed. A glibc 2.3.2 build host is required.
2. C++ headers need to go into lsb-build-base package, and lsb-build-c++ needs to go away completely. We still don't know what to do here; one thought is to just pick a set from a gcc version and copy them in. Generating these out of the DB is not in plan. For now, lsb-build-c++ supplies the headers. Question: is libsupc++.a needed?
3. C++ question: are we going to be able to accommodate anything other than gcc 3.3? A patch for 3.3 is partly accepted by upstream (for 3.4), but not all. A bigger patch for 3.2 is conceivable. 3.2 and 3.3 are rumored not to be compatible, even if 3.3 is built in 3.2-compatibility mode. Further, 3.4 changes the ABI (including the soversion). May need a migration strategy.
4. Looking for a way to support multiple concurrent lsb-build versions. This would help with transition to 2.0; also needed for bi-arch systems which may need to have two build environments. E.g., it's no good if both a 32-bit and 64-bit set of stub libs would install to exactly the same path.
5. Request: each package should be buildable for /usr/local (default), or other prefix if supplied. LSB build would use something like prefix=/opt/lsb. This mainly works for the four autobuilt test packages through INSTALL_ROOT; it does not work for all of the lsb-build packages which may have some paths built into them today (e.g. lsbcc has hard-coded paths, not a path set as a build-time target).
6. Curses issues: we supply curses.h and libncurses.so. Not having two names for both fools some software, should add libcurses.so as a symlink and ncurses.h as a symlink in lsb-build-base.
7. Decide what to do with lsbdev chroot bugs (Chris to look at these when he gets a chance). The chroot is untested since the package renaming/movig.
8. Inactive: investigation of lsbdev tools with a non-GCC compiler. Mats had some luck with this on ia32/ia64, but not without changes. Each compiler is likely to have slightly different build rules for the link step; lsbcc must know about these because by the time the compiler has inserted libs and other rules lsbcc is already out of the picture. E.g., for Intel's compiler, a "compatibility flag" -cxxlib-gcc is needed which lsbcc needs to know about or it will read it as -c; the compiler also requires some extra static libraries which lsbcc needs to know about too. To avoid exposing this to an existing build setup, it was necessary to hack lsbcc; long-term the compilers should support LSB-compatible building themselves. Mats was able to build appbat + extras + libbat, except for Celestia, with icc (8.0 beta) and a modified lsbcc (this was done when only openssl was in the libbat). One way to mitigate the hack-lsbcc problem, if the compiler doesn't support lsb building itself, would be to abstract the linker rules into a config file so different config files could be produced for other compilers.
9. lsb-rpm is not a maintained tool, it was a special build for the chroot to avoid importing extra libs. We would like to have an lsb package build tool, however - one that knew about lsb conformance rules and worked the same everywhere. Best would be if a maintained toool could have an lsb mode, instead of needing to maintain a separate tool or our own patchset. Can we make this happen for rpm (the most likely candidate)? Debian builds an lsb-rpm.
10. pkgchk needs to move towards a release. One round of test/fix was done recently. This is now considered a 2.0 deliverable.
11. Need resolution of lsb dependencies for multiple architectures (e.g. lsb-ppc32 and lsb-ppc64 on a ppc64 machine). Also for the lsb-release package, which provides the same function for non-rpm packages (affects release of lsbsi-lsb pkg) - need to define the syntax for this. In addition, a request was made to define, at the same time, how this will be extended for profiles (without necessarily defining the profile names yet - just define the naming scheme). See http://www.linuxbase.org/~mats/multi_arch.html for more discussion.

LSB Application Battery

1. Work was done to reduce the size of the Apache patch. Identifying some configure options that affect the "apache portable runtime" library have left the patch as essentially trivial.
2. postgresql was built successfully using 1.3 tools and will be tested with 2.0 snapshots. Only minor issues were found (these will be corrected in the 2.0 tools as they're base-spec variances there). A build of wine was attempted, identified some issues but is not able to proceed now without local patching.
3. A library battery is being built to get around certain build issues, usually vendors not shipping certain static libraries. Contents recently expanded to include freeglut (to help build Celestia, some distros don't ship libglut.a and some are now removing glut entirely for license reasons), freetype, libssl/libcrypto from openssl, jpeg, tiff and png. Build with 'make libbat' in the appbat tree. A library battery might also be needed for LSB 1.3; in general, glibc-2.3-based systems can't fully build the appbat, and most of the 1.3 certifications have been on such systems.
4. We can now build the current appbat with current tools, excepting Celestia in one situation, which may be a local tools issue. libtool has been an ongoing problem. libtool runs before lsbcc/lsbc++ called, so it may find a library file (xxx.la) in the wrong place, and use an explicit path to that library. Thus, having the libbat only helps sometimes. There's a way to call libtool so it looks the right places, still exploring whether it's enough, if dummy .la files are added to /opt/lsb/lib which indicate no shared lib.
5. Appbat now has a configuration script which tests for some key features before the build is kicked off. The configure script has introduced an interesting logical problem if libbat is not installed: it attempts to test for build dependencies. If dependencies that will be satisfied by libbat are not present, then configure will fail and thus there's no makefile to build the libbat with. This will not be a problem once prebuilt libbats are available for each architecture. A special makefile was recently added to enable libbat to build even if the configure has not completed; this needs testing.
6. A table of packages with new upstream versions is included below. Even after a round of uplifiting there's more to do so that appbat applications are as reflective of current upstream as possible on the release date. It is understoon that sometimes a new version introduces major new issues through the addition of large chunks of new functionality or a rewrite of a large section, may wish to avoid these (document why, in this case) if that's not an issue that drives the LSB forward. xpdf version 2.x requires Motif for the viewer, fooling with that proved non-trivial and does not currently advance the state of the LSB so will be left alone.
7. Renaming issue: appbat packages won't move to /opt/lsb. Putting Apache, for example, in /opt/lsb-apache might imply it's an officially supported build of Apache for LSB. Maybe /opt/lsb-appbat as a namespace for all of the appbat programs?
8. See library dependency graphs at http://www.linuxbase.org/~matsw. The dependency tool is in the LSB-futures web tree http://www.linuxbase.org/futures/identification/depends/lsbdepgraph.py.
9. An uplift to latest nALFS was considered, but will not be done now. A reason for considering this is that the xml in use is not valid according to external validators and thus restricts the tools available to work with it (e.g. the check/download tool can't use an xml parser to read the package information). However, the use of nALFS at all remains problematic when appbat is considered a build example to others, as they're unlikely to adopt yet-another-tool. Perhaps should return to rpm-based building, if we could reliably capture all the information to eliminate the reproducibility problems on ia32 that were a problem before. Revisit if we get some more resources to help. There's also the possibility of a tool which would dump the xml in a command-line type format.
10. Make the FVT capture-and-submit process more automated where feasible. Currently, you run all the FVTs manually and then assert in your certification application that you did so, and they passed. No results are submitted, and there's no automation. Minimal automation concept is to, where possible, have setup instructions, followed by one or more steps (possibly with further setup in between) that kick off a test and return a pass-fail result, so there's less visual evaluation to determine whether it worked or not. For some apps it's not feasible without a sophisticated test driver tool (xpaint, Celestia, etc.) - basically doesn't work if graphics are involved.
11. The appbat rpms will include a target LSB version as well as package version, encoded as "package-upstreamversion-lsbbuildnumber-lsb20.arch.rpm". Some distros are using similar naming schemes for packages now.
12. Old, needs resolving: some problems detected with xpaint on one platform. Question: does this happen with a native xpaint installed, or not - not sure if the app-defaults for xpaint are going to the right place (i.e., may pick up from native xpaint if one is installed, and not find one if the LSB version puts it in the wrong place and there's no native version).
13. Need an appbat package to use install_initd/remove_initd to install/remove startup scripts as an example.
14. Build Dependency tool - currently nALFS doesn't have a good way to represent build dependencies, may need to build a tool to help with this, esp. when appbat is used outside LSB project as an example. Appbat gets around this through its' configure script.
15. Add new representative sample applications, particularly C++. Candidates: openjade (already built, but not included in the core set)? ImageMagick (has a C++ binding Magick++)? mysql? openssh? ghostscript? lyx? Note: mysql built, openjade already added to appbat (but not in certification bundle). Added to investigation list during the 11Nov call: Mozilla Firebird, the Gimp, bochs, alpha player, flightgear, openvrml, vncserver. Most are C++; some are to test if the build infrastructure has improved enough (e.g. Mozilla was too hard to build in the past, is it better now? the Gimp is not C++ but does use the bottom layer - only - of Gnome: glib,gtk,gdk,gmodule).
16. Check all existing packages for FHS conformance. Possible areas of concern: if an X app-defaults file is installed, does the app find it? Are there /var/run scripts that are being put in /var/opt/pkgname/run instead? etc.
17. A suggestion was raised to have the appbat packages indexed by rpmfind. This is under discussion. One thought was that maybe the upstream maintainers ought to give permission. A second question was whether we should continue the build-with-nALFS, package-with-rpm policy, or whether we should try to return to building with rpm.
18. We've been approached about adding wine to the appbat. This would be i386-only, but it might be interesting to show problems in x libraries, and possibly in program execution areas. We'll revisit after it's actually been compiled in LSB mode.

Latest Status for Building of Application Battery

Latest Status for Building and Executing LSB Development Environment Checker

Appbat Packages with upgraded base

LSB Sample Implementation

1. bzip2 problem exposed by earlier upgrade to rpm 4.1 has been resolved.
2. xfree86 upgraded to 4.3, this allowed x86_64 to advance to phase3 in the build and should also help s390x. ppc64 may need a further patch, or it may be that the limited set we build (libraries only, no server, no cmds) is okay.
3. Also exprimented with unbundled X libs from freedesktop.org, these let us limit what to build to the exact set LSB needs. All of the X libraries have built; Mesa (for openGL) does not link the library correctly on x86_64, appears to be a lib64 issue.
4. Work remains on file and library cleanup, esp. in coreutils (the existing file cleanup was not carried forward from the old fileutils/textutils/sh-utils). A pass was done to identify "extra" shared libraries and binaries with a plan to remove unneeded ones from the lsbsi. The installed files for each package have been captured (there's an nALFS option for this) - need to schedule the work to do this cleanup
5. Platform build status 05 March:
   1. i386, ia64, s390, ppc32 built to completion prior to the xfree86 package upgrade (need to retest with this change).
   2. s390x had a configure change to gcc to avoid multilib (same as x86_64). It now fails building glibc in phase3.
   3. x86_64 fails building gettext in phase3
   4. ppc64 does not build bootstrap compiler; toolchain issue.
6. Remaining /lib64 issues: a few packages are installing into /lib or /usr/lib where they should not. Several such packages were fixed during the past week. These are not currently blocking the build but will do so eventually and will need cleanup. /lib64 platforms are not building biarch at this time - probably won't for LSB 2.0 if ever.
7. New questions: should the toplevel directory of the lsbsi reflect the architecture, to make it easier for two lsbsi's to be resident on a dual-architecture machine? It could be solved by unpacking the lsbsi in different directories, there's just a small risk a user won't remember to do that.
8. Previous test runs on Sample Implementation snapshots are now out of date; the new LSB 2.0 tests will begin to pilot by March 15 and we'll retest on all platforms that build to completion. Older results indicate that 1.3 tests did not show regressions except where the newer glibc (2.3.2) no longer includes a workaround for a test failure that was present in older (2.2.5) glibc versions.
9. The bootstrap and phase3 need to be verified for the correct build logic. Both install to an alternate root, but when actually running, these are using the standard root. Thus, they should not configure for a non-standard path; rather they should configure a standard path and then use DESTDIR or equivalent when installing. Some time has been spent on this and most of the questionable packages have been fixed, but should be reviewed again.
10. (Old). The lsbsi is still missing install_initd and remove_initd (it provides empty dummies). These are really failures to conform to the spec, but as there's no test they aren't flagged as such by the testing procedure. Still looking for volunteers to work on both a sample implementation and a test suite. An idea raised in the past might be worth considering: the sample could perhaps work on a non-standard location so that "cheaters" who try to install a startup script directly instead of using the required tool to install them could get detected by installing in the lsbsi.
11. The 'lsb' packages for all architectures need to be released for lsb 2.0. Note this package is now known as lsbsi-lsb, to emphasize it's only intended for the lsbsi. It's currently tagged 1.9, and this is what the lsbsi profile is expecting. This package supplies lsb_release, install_initd, remove_initd, and the proper dependencies. A complication is that because of profiles, and because of multi-arch, the answers a system can give may be not only complex, but may change over time. The simplest way to support that seems to be to have an information directory from which capabilities can be added/removed (/etc/lsb-release.d); the tool will interpret what's there.
12. Bryan Dumm did some work on creating an lsbsi-devel package. The thought is to restore those development tools used by phase 2 but which are not built for phase 3. This includes rebuilding glibc since the development pieces of glibc are removed from phase 3. There are some thoughts on redesigning the build procedure to make this easier. .

LSB-si Packages with upgraded base

LSB-si Standards Compliance

This section documents the known variances of the lsbsi from LSB version 1.3. 1.9 results are summarized above in the lsb-si section.

The lsbsi does not provide (or rather provides empty) required tools install_initd and remove_initd. These are not currently tested for, so do not show as failures.

The state of PAM in the lsbsi is still somewhat of an unknown, awaiting tests.

The following are the 44 known internationalization failures in the LSB-si running the 1.3.3 test suites. They show non compliance in the upstream packages diffutils (1), grep (3) and textutils (40). With the exception of the grep package failures, these are now waived via INT.010 (PR 0037) for LSB 1.3, no determination of the status of requiring these for 2.0 has yet been made. The grep issue is fixed in grep 2.5.1, but the release status of this version is uncertain at this time so remains unfixed. Some earlier failures which are also waived through INT are not listed here.

/tset/LI18NUX2K.L1/utils/diff/diff 2 FAIL
/tset/LI18NUX2K.L1/utils/egrep-tp/egrep-tp 5 FAIL
/tset/LI18NUX2K.L1/utils/fgrep/fgrep 5 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 1 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 2 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 3 FAIL
/tset/LI18NUX2K.L1/utils/grep-tp/grep-tp 5 FAIL
/tset/LI18NUX2K.L1/utils/join/join 3 FAIL
/tset/LI18NUX2K.L1/utils/join/join 4 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 1 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 3 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 4 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 5 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 6 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 8 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 9 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 10 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 11 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 12 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 13 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 14 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 15 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 16 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 24 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 25 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 26 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 27 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 28 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 29 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 30 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 31 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 32 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 40 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 41 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 42 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 43 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 44 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 45 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 46 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 47 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 48 FAIL
/tset/LI18NUX2K.L1/utils/unexpand/unexpand 1 FAIL
/tset/LI18NUX2K.L1/utils/uniq/uniq 2 FAIL
/tset/LI18NUX2K.L1/utils/uniq/uniq 3 FAIL

With runtime test 1.3.6 approved for certification, there was an additional failure introduced from the gettext package. This failure is waived by INT.007 (PR 0031).

/tset/LI18NUX2K.L1/utils/msgfmt/msgfmt 9 FAIL