Walk-Through: Compiling the FreeBSD Base System, Using Source Code Retrieved via Git (Draft 2)

Introduction - Towards a Trusted Computing Platform Derived of FreeBSD

Beginning with the FreeBSD base system, it may be possible to develop a complete platform for trusted computing, with a complete, documented chain of responsibility from source code to application. Such a trusted computing platform -- as initially may extend of the FreeBSD base system  -- may be furthermore extended of the work of the BSSSD project, and the work of any discrete number of additional software development projects, such that the collected software may be applied as to provide fundamental services and userspace applications for FreeBSD as a trusted computing platform. Such a platform may be applied as onto any discrete number of machine architectures for which there is a port of the FreeBSD base system and a corresponding port of each of the collected set of additional software applications.

In a low-level functional regards, such a trusted computing platform may be developed in application of any discrete number of existing features as presently developed in the FreeBSD kernel and broader base system, as well as of any discrete, additional number of userspace applications.  In its development, furthermore, a FreeBSD trusted computing platform may serve as keynote for development of additional kernel  features and new userspace applications.

The author will presently admit the author's own bias, as with regards to a small number of furthter thesis concepts, viz a viz  the author's own "To Do" and/or "Wish List" in many regards of computing:

• Development of networked userspace applications using CORBA for communications infrastructure, towards integrating the FreeBSD netgraph architecture and I2C daemon for communications among CORBA object request broker (ORB) processes such as may be implemented on a per-interface bases and jailed for purpose of system security, then in an an applciation of the FreeBSD process jail model.
• Applications of transport layer security (TLS) and authentication frameworks such as SSL and Kerberos, in a CORBA usersrpace infrastructure
• Scaling of enterprise service models for application of services in small office and home office (SOHO) commercial environments
• Licensed radio frequency communications on amateur radio bands
• Computing not indoors
• Human computer interface (HCI) design in a classical mode, focusing on the Common Lisp Interface Manager (CLIM) and the hardware interfaces available for applications in contemporary environments for desktop and mobile computing 
• Knowledge development, knowledge modeling, and authenticated knowledge sharing, within frameworks of the original Knowledge Interchange Format (KIF), Common Logic, the Resource Description Framework (RDF), the Web Ontology Language (OWL), and corresponding service models such as SPARQL 
• Native-coded Common Lisp implementations, including Steel Bank Common Lisp (SBCL) and the original CMU Common Lisp (CMUCL)
• Common Lisp implementations making a substantial application, each, of C toolkits, focusing on Clozure Common Lisp (CCL), Embeddable Common Lisp (ECL), and Man Kai Common Lisp (MKCL)
• FPGA programming and studies of the architecture of the original MIT CADR Lisp Machine for possible extension with contemporary bus/interface models, as well as factory-programmed microcontroller architectures, and repgorammable microcontroller archiectures
• Potential applications of SCADA communications infrastructure models for trusted communications among ground stations and nanosatellites in Low Earth Orbit (LEO) and Lagrangian orbits
• The Metaobject Framework (MOF) for its applications in the combined Object Management Architecture (OMA) and Model Driven Architecture (MDA) each as a conceptual frameworks for best practice with model-driven tools, developed and published of the Object Management Group (OMG). 
• Application of the MOF to IDL (MOF2I) transformation, for integration of MOF metamodels, corresponding models, and application runtime data within CORBA architectures
•  The Metaobject Framework (MOF) as extended with an abstract graphical syntax corresponding to entities defined in the Unified Modeling Language (UML) furthermore extensible with UML profiles inlcuding MARTE, SPEM, and UML4SysML
• The Metaobject Framework (MOF) as extended with a metamodels for each of RDF, OWL, Common Logic, and ISO Topic Maps, in the ODM specification
• Archaeologcial studies as may be conducted towards developing conceptual genealogies illustrating progressive developments among correlated, discrete species of structures and systems

In the author's own opinion, considering the previous outline of thesis concepts, FreeBSD may be made to a a most apparent application, as in regards to CORBA services and HCI designs. The singular nature of the FreeBSD base system, moreover, as a fundamental quality of a trusted computing platform, it may serve to greatly aid in the development of any of those or any similar thesis concepts, in any academic and technical productions.


Before Boot 0: Building the Kernel and Base System


The following article is presented as a small effort by the editor, for publishing a small number of assorted notes, as with regards to building a FreeBSD base system and updating the existing FreeBSD base system installed on a FreeBSD host. The following example is developed with FreeBSD 10, beginning with a host on which the EFI-enabled baseline FreeBSD version 10.1-RELEASE was installed from DVD media.

For brevity, and in candor, this article is written with an assumption as that not many people besides the author may be reading this article. Therefore, this article leaves out any manner of further or more detailed introduction or overview text, assuming:

• That the reader is familiar with the TTY shell environment of FreeBSD version 10.1-RELEASE
• That the reader is familiar with the Git software change and configuration management (SCCM) tool, at least inasmuch as a familiarity with regards to essential clone and branch operations in Git, secondly familiar with Git revision specifiers
• That the reader has a user account at GitHub
• That the reader has generated a suitable private key and registered its corresponding public key in the user's account configuration at GitHub
• That the reader may be familiar with the screen shell command and a number of more common shell commands such as su and chown
• That there is a host named builder, configured with a primary non-root user 'me' on the builder host -- such that user 'me' has UID 1000 
• That the user having UID 1000 will be retrieving the latest source code updated for FreeBSD 10, for compilation on the builder host
• That the shell command eval $(ssh-agent) was performed at time of user login, to an effect that an ssh-agent process is initialized, is running in the background of the controlling terminal, and that the SSH agent will have published the bindings for a corresponding set of environment variables, for evaluation during the login process. Any subsequent child process forks may then read those environment variables, contingent to communicating with the SSH agent process.

The following shell session begins with the user 'me'  logged in on a terminal on a host in which FreeBSD 10.1-RELEASE is installed. Previous to the main work of the script, the user calls ssh-add to ensure that the user's default private key on the builder host will be read and stored in memory by the SSH agent process. This procedural step is performed at the user's discretion, essentially as a matter of functional convenience, previous to downloading the base system's source code via GitHub.

Subsequently, a screen session is initialized, such that the shell process forked by the screen session will inherit the shell environment of the earlier shell process -- thus, including any environment variables as may be necessary for further forked processes to communicate with the background  SSH agent process.

Lastly, the host's existing /usr/src source tree is moved to a temporary location on the filesystem, such that the original source tree may then be stored in a tape archive and optionally removed from the filesystem. A new /usr/src/ directory is then created, with ownership by the user 'me', having UID 1000. The user  'me' then exits the superuser process.

Subsequently, the 'me' user downloads the base system source code repository via GitHub. The FreeBSD releng/10.1 branch is retrieved, firstly, such that that branch, as locked at time of retrieval, will then be available in the local base system's source code repository on the builder host. This step may be performed as in the interest of a possibility that that the original, upstream 10.1-RELEASE source tree might be needed for any manner of later reference, in development tasks.

The /usr/src directory, in its end state, will contain the source code of the FreeBSD stable/10 branch.

Example: Shell session for checkout of FreeBSD base system via the FreeBSD Project's freebsd repository at GitHub

[me@builder ~/]$ uname -a
FreeBSD builder 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
[me@builder ~/]$ cd /usr
[me@builder /usr/]$ ssh-add
... interaction ...
[me@builder /usr/]$ screen -S base
... screen session ...
[me@builder /usr/]$ su
[root@builder /usr/]# mv src src_FreeBSD-$(uname -r)_ISO_src
[root@builder /usr/]# mkdir src
[root@builder /usr/]# chown 1000 src
[root@builder /usr/]# exit
[me@builder /usr/]$ cd src && git init
... output from git ...
[me@builder /usr/src/]$ git remote add origin git@github.com:freebsd/freebsd.git
[me@builder /usr/src/]$ git pull origin releng/10.1 # 10.1-RELEASE (frozen)
... output from git ...
[me@builder /usr/src/]$ git pull origin stable/10 # development for FreeBSD 10-STABLE
... output from git ...
[me@builder /usr/src/]$ git reset hard && git clean -fd # cleanup after merge to implicit local master branch
... output from git ...
[me@builder /usr/src/]$ git checkout stable/10
Branch stable/10 set up to track remote branch stable/10 from origin.
Switched to a new branch 'stable/10'
[me@builder /usr/src/]$ git branch
  master
* stable/10
[me@builder /usr/src/]$ git branch delete master

At that point, the directory /usr/src contains the latest updated source code from the upstream source repository at GitHub, with the source tree of the upstream branch stable/10 then checked out into /usr/src/.

As an orthogonal note, Git will have created an additional master branch during the initial pull process. In that script, as illustrated, the Git reset and clean commands were applied as to clean up the local source tree as to prevent a collision between the implicit local master branch -- as was initialized from the releng/10.1  upstream branch -- a collision that would otherwise occur when  the upstream stable/10 branch is subsequently fetched and checked out into the local source tree. The final shell command in the script will have deleted that implicit master branch -- otherwise, it might cause any number of further revision conflicts, during the process of branch checkout. 

In order to prevent the initial collision, moreover, the git fetch command could be applied as instead of the second git pull command in the script. The git branch delete   master command could then be performed before the final git checkout, thus ensuring that the local source tree would then contain an exact representation of the upstream source tree, with no initial collisions in source files.

The builder host may then be prepared for compiling the base system.


Notes: Optimizing for Build

To the author's best assumption, the LLVM C compiler toolchain may be installed  as a part of the FreeBSD base system.

Previous to this task, the Poudriere build automation tool had been installed on the builder host. Poudriere was subsequently applied for building a small number of FreeBSD ports, using the Hardpan autoport shell scripts,  It may be assumed that the C compiler toolchain, at the time of this task, was correctly configured for the host.

In order to ensure that the LLVM C compiler toolchain will produce object files such that may be optimized for the microprocessor installed o the builder host and for similar microprocessors in hosts on the local area network (LAN), the following configuration properties had been specified in /etc/make.conf.

CPUTYPE?=core2
CFLAGS+=        -msse3
CXXFLAGS+=      -msse3

COPTFLAGS?= -O -pipe -j4
FORCE_MAKE_JOBS=yes
MAKE_JOBS_NUMBER=4

The previous configuration should furthermore serve to ensure that the C compiler will be applied in parallel jobs, when using the LLVM C compiler toolchain as in a manner independent of other build automation tools. Consulting the documentation for build automation tools such as Poudriere, the respective build automation tool may offer a number of additional options for controlling a build process.

Notes: The Build Process - Kernel Build

... TBD ...

... sidebar; kernel configuration profiles ...

Notes: The Build Process - Base System Build

... TBD ...

Notes: Installing via NFS

Courtesy of advice presented in the FreeBSD community support channel on the FreeNode IRC chat network, the author has learned of a recommendation. in effect, as that files compiled of 'make buildworld' and any static, host-level files of the FreeBSD base system may be installed for system update over a local area network, using a networked filesystem (NFS) share mounting /usr/src. Such a configuration may serve to allow for 'make installworld' to be called on each respective host where the NFS share is  mounted. This, in turn, may serve to allow for any manner of host-local change management to be performed for configuration files on the respective installation host, during the make installworld process.

It may be furthermore advisable that the NFS share would not be published to the network, until after the initial build process will have successfully completed.



Bibliography

• Stokely, Murray. FreeBSD Release Engineering
• Fuller, Matthew D. What's a 'Base System'?. 
• The FreeBSD Documentation Project. FreeBSD Handbook
• Referencing Ch. 9,  Configuring the FreeBSD Kernel
• /usr/share/examples/etc/make.conf
• /usr/src/share/mk/bsd.cpu.mk
• Andree, Matthias. Can make -j be used for ports?
• FreeBSD File Formats Manual. make.conf -- system build information. FreeBSD 10.1-RELEASE
• Additional manual pages (informative)