Sample Implementation, Application Battery and LSB Build Status

Week ending Jan 9, 2004

LSB Build Tools

  1. lsbdev 1.9 snapshots are now being built assuming a C++ shared library. lsb-build-c++ is still needed to supply the headers. Autobuilder is running for six architectures; AMD64 needs to be enabled. lsb-build-c++ is not autobuilt; it requires lsbc++ to compile and is done by hand. Another quirk is that libchk now requires a linked binary, which it uses to detect the library locations it's going to test. This currently links against lsb stub libraries. Warnings: a glibc 2.3.2 build host is required. lsb stub libraries (lsb-build-base) needed to link libchk.
  2. C++ headers need to go into lsb-build-base package, and lsb-build-c++ needs to go away completely. We still don't know what to do here; one thought is to just pick a set from a gcc version and copy them in. Generating these out of the DB looks daunting.
  3. Are we going to be able to accommodate anything other than gcc 3.3? 3.4 changes the ABI (including the soversion) and so it out; 3.2 looks like it's missing too much. Is that too restrictive to be useful? A patch to the linker script for 3.2 exists, and with it 3.2 libs are pretty good, but that would require a re-release of those libraries by older distros, unlikely to happen. Even 3.3 is missing some symbols from the ABI set, so it looks like the target is going to have to be yet-to-be-release distributions.
  4. Looking for a way to support multiple concurrent lsb-build versions. Needed for internal transition phase to 1.9/2.0; also needed for bi-arch systems which may need to have two build environments. E.g., it's no good if both a 32-bit and 64-bit set of stub libs would install to exactly the same path.
  5. Request: each package should be buildable for /usr/local (default), or other prefix if supplied. LSB build would use something like prefix=/opt/lsb. This mainly works for the four autobuilt test packages through INSTALL_ROOT; it does not work for all of the lsb-build packages which may have some paths built into them today (e.g. lsbcc has hard-coded paths, not a path set as a build-time target).
  6. Curses issues: we supply curses.h and libncurses.so. Not having two names for both fools some software, should add libcurses.so as a symlink and ncurses.h as a symlink in lsb-build-base.
  7. libchk build currently depends on an installed set of stubs for linking the dummy program libchk uses to derive the paths to the real runtime libraries. Consider a scheme where this build can be more self-contained (perhaps build a local copy of stubs and link with them).
  8. Decide what to do with lsbdev chroot bugs (Chris to look at these when he gets a chance). The chroot is untested since the package renaming/movig.
  9. Investigation of lsbdev tools with a non-GCC compiler. Mats had some luck with this on ia32/ia64, but not without changes. Each compiler is likely to have slightly different build rules for the link step; lsbcc must know about these because by the time the compiler has inserted libs and other rules lsbcc is already out of the picture. E.g., for Intel's compiler, a "compatibility flag" -cxxlib-gcc is needed which lsbcc needs to know about or it will read it as -c; the compiler also requires some extra static libraries which lsbcc needs to know about too. To avoid exposing this to an existing build setup, it was necessary to hack lsbcc; long-term the compilers should support LSB-compatible building themselves. Mats was able to build appbat + extras + libbat, except for Celestia, with icc (8.0 beta) and a modified lsbcc (this was done when only openssl was in the libbat). One way to mitigate the hack-lsbcc problem, if the compiler doesn't support lsb building itself, would be to abstract the linker rules into a config file so different config files could be produced for other compilers.
  10. lsb-rpm is not a maintained tool, it was a special build for the chroot to avoid importing extra libs. We would like to have an lsb package build tool, however - one that knew about lsb conformance rules and worked the same everywhere. Best would be if a maintained toool could have an lsb mode, instead of needing to maintain a separate tool or our own patchset. Can we make this happen for rpm (the most likely candidate)?
  11. pkgchk needs to move towards a release. One round of test/fix was done recently. This is now considered a 2.0 deliverable.
  12. Need resolution of lsb dependencies for multiple architectures (e.g. lsb-ppc32 and lsb-ppc64 on a ppc64 machine). Also for the lsb-release package, which provides the same function for non-rpm packages (affects release of lsbsi-lsb pkg) - need to define the syntax for this. In addition, a request was made to define, at the same time, how this will be extended for profiles (without necessarily defining the profile names yet - just define the naming scheme). See http://www.linuxbase.org/~mats/multi_arch.html for more discussion.

LSB Application Battery

  1. We can now build the current appbat with current tools, excepting Celestia in some situation (investigate).
  2. Appbat now has a configuration script which tests for some key features before the build is kicked off. The configure script has introduced an interesting logical problem if libbat is not installed: it attempts to test for build dependencies. If dependencies that will be satisfied by libbat are not present, then configure will fail and thus there's no makefile to build the libbat with. This will not be a problem once prebuilt libbats are available for each architecture.
  3. A table of packages with new upstream versions is included below. The focus is on C++ apps at the moment; Celestia and groff have been uplifted this week to the new versions indicated. Other uplifts are at the discretions of the team: try out the new versions as time permits, and make a decision. Sometimes a new version introduces major new issues through the addition of large chunks of new functionality or a rewrite of a large section, may wish to avoid these (document why, in this case) if that's not an issue that drives the LSB forward. xpdf version 2.x requires Motif for the viewer, fooling with that proved non-trivial and does not currently advance the state of the LSB so will be left alone.
  4. Renaming issue: appbat packages won't move to /opt/lsb. Putting Apache, for example, in /opt/lsb-apache might imply it's an officially supported build of Apache for LSB. Maybe /opt/lsb-appbat as a namespace for all of the appbat programs?
  5. A library battery is being built to get around certain build issues, usually vendors not shipping certain static libraries. Contents as of now are freeglut (to help build Celestia, some distros don't ship libglut.a and some are now removing glut entirely for license reasons), freetype, and libssl/libcrypto from openssl. It's possible to type ''make libbat'' in the appbat tree to build this. A library battery might also be needed for LSB 1.3; in general, glibc-2.3-based systems can't fully build the appbat, and most of the 1.3 certifications have been on such systems.
  6. See library dependency graphs at http://www.linuxbase.org/~matsw. The dependency tool is in the LSB-futures web tree http://www.linuxbase.org/futures/identification/depends/lsbdepgraph.py.
  7. An uplift to latest nALFS was considered, but will not be done now. A reason for considering this is that the xml in use is not valid according to external validators and thus restricts the tools available to work with it (e.g. the check/download tool can't use an xml parser to read the package information). However, the use of nALFS at all remains problematic when appbat is considered a build example to others, as they're unlikely to adopt yet-another-tool. Perhaps should return to rpm-based building, if we could reliably capture all the information to eliminate the reproducibility problems on ia32 that were a problem before..
  8. Make the FVT capture-and-submit process more automated where feasible. Currently, you run all the FVTs manually and then assert in your certification application that you did so, and they passed. No results are submitted, and there's no automation. Minimal automation concept is to, where possible, have setup instructions, followed by one or more steps (possibly with further setup in between) that kick off a test and return a pass-fail result, so there's less visual evaluation to determine whether it worked or not. For some apps it's not feasible without a sophisticated test driver tool (xpaint, Celestia, etc.) - basically doesn't work if graphics are involved.
  9. The appbat rpms will now include a target LSB version as well as package version, encoded as "package-upstreamversion-lsbbuildnumber-lsb20.arch.rpm". Some distros are using similar naming schemes for packages now.
  10. Old, needs resolving: some problems detected with xpaint on one platform. Question: does this happen with a native xpaint installed, or not - not sure if the app-defaults for xpaint are going to the right place (i.e., may pick up from native xpaint if one is installed, and not find one if the LSB version puts it in the wrong place and there's no native version).
  11. Need an appbat package to use install_initd/remove_initd to install/remove startup scripts as an example.
  12. Build Dependency tool - currently nALFS doesn't have a good way to represent build dependencies, may need to build a tool to help with this, esp. when appbat is used outside LSB project as an example.. A solution might be to capture these ourselves in libbat - might solve short-term need.
  13. Add new representative sample applications, particularly C++.. Candidates: openjade? ImageMagick (has a C++ binding Magick++)? mysql? openssh? ghostscript? lyx? Note: mysql built, openjade already added to appbat (but not in certification bundle). Added to investigation list during the 11Nov call: Mozilla Firebird, the Gimp, bochs, alpha player, flightgear, openvrml, vncserver. Most are C++; some are to test if the build infrastructure has improved enough (e.g. Mozilla was too hard to build in the past, is it better now? the Gimp is not C++ but does use the bottom layer - only - of Gnome: glib,gtk,gdk,gmodule).
  14. Check all existing packages for FHS conformance. Possible areas of concern: if an X app-defaults file is installed, does the app find it? Are there /var/run scripts that are being put in /var/opt/pkgname/run instead? etc.
  15. A suggestion was raised to have the appbat packages indexed by rpmfind. This is under discussion. One thought was that maybe the upstream maintainers ought to give permission. A second question was whether we should continue the build-with-nALFS, package-with-rpm policy, or whether we should try to return to building with rpm.

LSB Sample Implementation

  1. A round of package uplifiting has been done, including the three major pieces: new gcc, new glibc, and replacing fileutils/textutils/sh-utils with coreutils (some work remains on coreutils to remove non-LSB files). See table below. This build is now indicating version 1.9. glibc is built with tls (thread-local-storage), but not with nptl. Is tls okay? One thought is an alternate phase producing an nptl-enabled glibc that could be used for experimentation.
  2. A test run with current cvs (1.9) turned up 53 failures. The new failures are with tcgetattr 1,2 and i_flag 6,7,8,9, which we believe have been seen elsewhere. There's also one new msgfmt (msgfmt 5) failure, and one easily fixable bin-tc failure. This in addition to the 45 listed below as known 1.3 failures.
  3. Sample Implementation design remains to complete. At the moment it looks like AMD64 will be a pure 64-bit platform similar to Itanium but with ia32 libs; powerpc64 will be a 32-bit platform (should be able to be a copy of powerpc32) with 64-bit libraries. Need info on s390x. This implies that "configure" needs to be architecture-specific, as it needs to supply --libdir=/usr/lib64 (for example) on some platforms. A hope is that the toolchain will support building both 32 and 64 code without having parallel versions; this would simplify the intermediate phase. Indications are that gcc and binutils are okay with this. Are there other tools that are architectue-sensitive?
  4. The bootstrap and phase3 should be checked for the correct build logic. Both install to an alternate root, but when actually running, these are using the standard root. Thus, they should not configure for a non-standard path; rather they should configure a standard path and then use DESTDIR or equivalent when installing. A number of packages are known to configure for a nonstandard prefix instead.
  5. (Old). The lsbsi is still missing install_initd and remove_initd (it provides empty dummies). These are really failures to conform to the spec, but as there's no test they aren't flagged as such by the testing procedure. Still looking for volunteers to work on both a sample implementation and a test suite. An idea raised in the past might be worth considering: the sample could perhaps work on a non-standard location so that "cheaters" who try to install a startup script directly instead of using the required tool to install them could get detected by installing in the lsbsi.
  6. The 'lsb' packages for all architectures need to be released for lsb 2.0. Note this package is now known as lsbsi-lsb, to emphasize it's only intended for the lsbsi. This package supplies lsb_release, install_initd, remove_initd, and the proper dependencies. A complication is that because of profiles, and because of multi-arch, the answers a system can give may be not only complex, but may change over time. The simplest way to support that seems to be to have an information directory from which capabilities can be added/removed (/etc/lsb-release.d); the tool will interpret what's there.
  7. Bryan Dumm did some work on creating an lsbsi-devel package. The thought is to restore those development tools used by phase 2 but which are not built for phase 3. This includes rebuilding glibc since the development pieces of glibc are removed from phase 3.

Appbat Packages with upgraded base

Package Appbat
Version		 Upstream
Version
celestia 1.2.5 1.3.0 Done
groff 1.17.2 1.18.1 Done. 1.19 is available from maintainer.
httpd 2.0.43 2.0.48 2.0.43 patch does not apply cleanly. 
Python 2.2.2 2.3.2 Some new porting issues.
rsync 2.5.5 2.5.6
samba 2.2.7 2.2.8a/3.0.0  
tcl/tk 8.3.4 8.4.4  
expect 5.38.0 5.39.0  
xpaint 2.6.2 2.7.0  
xpdf 1.01 2.03 xpdf 2.x requires Motif or lesstif. Neither works; both expose missing interfaces in libX11/libXt.
OpenSP 1.5 1.5.1 Done. (OpenSP needed for openjade)

LSB-si Packages with upgraded base

Package LSBSI
Version		 Upstream
Version		 Comments
at 3.1.7 3.1.8 Done
binutils 2.13.2 2.14 2.14 is done; 2.15 is close from upstream
bison 1.35 1.875  
coreutils  Note 5.0 Partly done (replacing fileutils 4.1, textutils 2.1, sh-utils 2.0). Needs work on program removal.
db 3.2.9 3.3.11/4.x Question whether Berkeley DB is needed at all (rpm includes a private copy and is statically linked)
diffutils 2.8.1 2.8.4 Done 
e2fsprogs 1.32 1.34 Done
file 3.39 4.07 Done
gawk 3.1.1 3.1.3 Done 
gcc/g++ 3.2.2 3.3.2 Done
gettext 0.11.5 0.12.1 Done
glibc 2.2.5 2.3.2 Done
groff 1.18.1 1.19 Done
kbd 1.06 1.08 Is kbd needed at all?
lfs-bootscripts 1.10 1.12  
man 1.5k 1.5m2 Done
man-pages 1.53 1.64 Done 
mktemp 1.4 1.5 Done 
modutils 2.4.22 2.4.26 See http://www.kernel.org/pub/linux/utils/kernel/modutils/v2.4/README on post-2.4.23 issues
pax 3.0 3.1 Done 
psmisc 21 21.3 Done 
rpm 3.0.6 4.1 version was previously "kept low" intentionally
sed 4.0.5 4.0.8 Done
shadow 20001016 4.0.3 previous uplift attempt failed one test (believe patch has been identified)
sysvinit 2.84 2.85 Done
texinfo 4.3 4.6 Done 
util-linux 2.11y 2.11z 2.11z done. 2.12 is near release
X11 libraries 4.2.0 4.3.0 New layout makes this problematic (have to grab everything)

LSB-si Standards Compliance

This section documents the known variances of the lsbsi from LSB version 1.3. 1.9 results are summarized above in the lsb-si section.

The lsbsi does not provide (or rather provides empty) required tools install_initd and remove_initd. These are not currently tested for, so do not show as failures.

The state of PAM in the lsbsi is still somewhat of an unknown, awaiting tests.

The following are the 44 known internationalization failures in the LSB-si running the 1.3.3 test suites. They show non compliance in the upstream packages diffutils (1), grep (3) and textutils (40). With the exception of the grep package failures, these are now waived via INT.010 (PR 0037) for LSB 1.3, no determination of the status of requiring these for 2.0 has yet been made. The grep issue is fixed in grep 2.5.1, but the release status of this version is uncertain at this time so remains unfixed. Some earlier failures which are also waived through INT are not listed here.

/tset/LI18NUX2K.L1/utils/diff/diff 2 FAIL
/tset/LI18NUX2K.L1/utils/egrep-tp/egrep-tp 5 FAIL
/tset/LI18NUX2K.L1/utils/fgrep/fgrep 5 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 1 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 2 FAIL
/tset/LI18NUX2K.L1/utils/fold/fold 3 FAIL
/tset/LI18NUX2K.L1/utils/grep-tp/grep-tp 5 FAIL
/tset/LI18NUX2K.L1/utils/join/join 3 FAIL
/tset/LI18NUX2K.L1/utils/join/join 4 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 1 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 3 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 4 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 5 FAIL
/tset/LI18NUX2K.L1/utils/pr/pr 6 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 8 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 9 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 10 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 11 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 12 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 13 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 14 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 15 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 16 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 24 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 25 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 26 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 27 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 28 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 29 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 30 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 31 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 32 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 40 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 41 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 42 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 43 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 44 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 45 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 46 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 47 FAIL
/tset/LI18NUX2K.L1/utils/sort/sort 48 FAIL
/tset/LI18NUX2K.L1/utils/unexpand/unexpand 1 FAIL
/tset/LI18NUX2K.L1/utils/uniq/uniq 2 FAIL
/tset/LI18NUX2K.L1/utils/uniq/uniq 3 FAIL

With runtime test 1.3.6 approved for certification, there was an additional failure introduced from the gettext package. This failure is waived by INT.007 (PR 0031).

/tset/LI18NUX2K.L1/utils/msgfmt/msgfmt 9 FAIL