Installed Fedora-Scientific 17. Checked out Scilab. Demos impressive. #scilab #fedora. Thanks Scilab packagers! twitter.com/echorand/statu…

— Amit Saha (@echorand) May 25, 2012

To state the obvious,  If you are not a KDE person, you may go ahead and install GNOME3, XFCE, LXDE witth yum, like so:

# yum group install “GNOME Desktop Environment”

# yum group install xfce

# yum group install lxde

Feel free to join us on SciTech SIG.

Support for building Live images: I attempted to use livemedia-creator (which is going to be THE tool from F18+), but unfortunately ran into issues which prevented me from building images. So for now, I have implemented this feature using ‘livecd-creator’. The Kickstart file (flattened) needs to be specified and other details such as architecture, any extra packages to be pulled from Koji, etc. The specifications are specified via the config/live.conf file.

The User specifications are now completely via .conf files: Myrationale behind that in the first place was that since this code is really going to serve as the ‘backend’, command line arguments could be done away with. But, even if we want to use this as standalone, specifying .conf files is fine as well. (We will see what happens with this after discussing with my mentors). Here is a brief description of the config files in config/

• imagebuild.conf: type of image, architecture, staging area (to be explained later) and email (for notification)
• boot.conf: configuration for boot.iso images
• repoinfo.conf: repository configuration required for the above
• pungi.conf: configuration for DVD images
• live.conf: configuration for live images

The kickstart files if needed are to be placed in the kickstarts/ sub-directory. To use it, you will need to ‘cd’ into ‘image_builder’ directory and run ‘$python run_imagebuild.py’ after setting up the appropriate config files in image_builder/config and kickstart files in kickstarts/
if any.

Support for copying images: I have also now enabled support for copying the images to a ‘staging area’ as mentioned earlier. I assume a passwordless login setup and hence do a ‘scp’ once the desired image has been created. This is how you would use it standalone.

Now, as a first step towards being able to distribute build jobs to different node, I have also now added simple support for carrying out the build process on a different host. This is done by the file
delegate.py.

Here is what it does:

• Assumes that the config/ and kickstart/ files have been correctly setup by the web-form handler or manually.
• Then copies these files to the image_builder/ directory
• It creates a .tar archive of the image_builder
• Then it reads the appropriate node (architecture) from the nodes.conf file and also retrieves the working location specified.
• The .tar file is then ‘scp’-ed to the appropriate location specified
• Then runs the run_imagebuild.py script on the build node by ‘SSHing’
• The image is then automatically transferred to the staging area
  specified as earlier.

To try this feature, simply setup nodes.conf file correctly and the config/ and kickstarts/ in data/ and run $sudo python delegate.py

Note that the specified nodes should have all the dependencies installed, such as lorax, koji (setup correctly), pykickstart and livecd-creator.

[1] https://github.com/amitsaha/gsoc2012_fbs

As you can see (on opening the screenshot), there is missing header error from ‘/var/www/html/owncloud/apps/files_odfviewer/appinfo/app.php’. This post here told me that it could be due to a empty line. So that;s it. I went to the file and removed the line no. 8 which was an empty line! And it works fine.

Couple of days back, I added the support for creating a DVD iso (using ‘pungi’). Basically the code requires a pungi.conf file to be specified. For example, here is a sample pungi.conf:


[DEFAULT]
name=Fedora
ver=17
arch=i686
flavor=Desktop
destdir=/tmp/pungi_op
cachedir=/var/cache/pungi
bugurl=http://bugzilla.redhat.com
nosource=1
sourceisos=0
force=1
stage=all
config=fedora-install-fedora.ks

#Specify a working directory
workdir=/tmp/pungi_work
# specify packages via NVR
nvr=
# specify packages via Build IDs (separated by a semicolon)
bid=314456


If you are familiar with pungi, you will notice the resemblance of the configuration options with the pungi command line options. The code basically reads this configuration file and fires pungi appropriately.

As you can see, the last two options allow you to specify more recent builds of packages (via NVR and Build IDs) which are not yet available in any of the release repositories and include them instead of the less recent ones. Since pungi requires a kickstart file to be specified, I update the kickstart file by adding the side repository URL to the list of repositories specified (script here at [1]). (I have hit a problem with this step: https://www.redhat.com/archives/kickstart-list/2012-May/msg00002.html)

The updated code is now in the git repository [2] and a sample command line to build a DVD iso would be: python imagebuild.py -t dvd -a i686 (after you have created the appropriate pungi.conf file). You can also use the run_imagebuild shell script for the same.

[1] https://github.com/amitsaha/gsoc2012_fbs/blob/master/scripts/ks_addrepo.py
[2] https://github.com/amitsaha/gsoc2012_fbs/tree/master/image_builder

Thoughts:

I am beginning to think of modifying the earlier boot.iso code to read in a configuration file instead of command line arguments, like I have done for the DVD iso. This is keeping in mind the fact this code will really be the backend of the web-based build service. Hence, .conf files created at the web based frontend and sent to this image builder code would be a good way to go about it, me thinks. No? We will see.

Next up will be support for creating Live media.

This command line spawns the build process of a Fedora 17 boot.iso with a number of extra packages (specified via their NVR or build IDs):

$ sudo python imagebuild.py -t boot -a i386 -o image_op1 -w image_work -p fedora -r 17 -v 1 -nvr 'anaconda-17.26-1.fc17' -bid '318281' '311809'

downloading packages
( 1/592) [100%] downloading GConf2-3.2.5-1.fc17.i686.rpm
( 2/592) [100%] downloading ModemManager-0.5.2.0-1.fc17.i686.rpm
( 3/592) [100%] downloading NetworkManager-0.9.4.0-7.git20120403.fc17.i686.rpm
( 4/592) [100%] downloading NetworkManager-glib-0.9.4.0-7.git20120403.fc17....
( 4/592) [100%] downloading NetworkManager-glib-0.9.4.0-7.git20120403.fc17....
.
.


Finally, a boot.iso is created. Next, I plan to inegrate package retrieval from Koji via other methods. And then, support for creating Live images and DVD images. And then, the REST API/Web based service.

Some of the scripts I was experimenting with to pull packages are here [2].

[1] https://github.com/amitsaha/gsoc2012_fbs/tree/master/image_builder
[2] https://github.com/amitsaha/gsoc2012_fbs/tree/master/scripts

1. lorax
2. pungi
3. livemedia-creator

Having had a basic idea of how lorax works,  I then proceeded to use pylorax to create a boot.iso by building upon Tim Flink’s image building code he had sent me during our project discussions. A build is now in progress as I write this.

My next plan is to integrate the creation of the side repository from extra packages retrieved from Koji so that newer builds of packages are included in the boot.iso.

Also, start using configuration files for specifying the repository/mirror information, architecture, release, etc. By next week, I should have this code in my git hub.

Once I setup Koji, I started playing around with the client code that Tim Flink had sent me earlier. I adopted Tim’s code to create a script to download RPM’s from Koji and create a side repository with them. The Python code is called: dl_package.py.

Next, I wanted to have a script which would download the latest build of a package for a particular tag from Koji. For this, I used koji_utils.py from Autoqa‘s code. The code is called: dl_latest_build.py. As of now, this script just downloads the RPMs for each of the tags.

Both these scripts are available here: https://github.com/amitsaha/gsoc2012_fbs/tree/master/scripts

Here’s the abstract:

On-Demand build service seeks to build Live and installation images for developers, testers and consumers of Fedora Linux. During the testing of Fedora releases, test images are often useful as smoke tests before full TC/RC composes, as baselines for specific test days or for automated installation testing in AutoQA. The idea is to make an on-demand Web-based build service which users/developers can use to make custom Fedora based distributions so that command-line wrangling can be minimized.

More details is available here. I shall be blogging more about my progress here. Many thanks to my mentor Tim Flink for his support during the application.

Congratulations to all the other projects accepted as part of the Fedora project.

Virtual Appliances using BoxGrinder

Now, I could definitely install one of the available Linux distributions onto each of my virtual machine (say, 3). But, why not go for something more barebone, such as Virtual Appliances. I had also been playing around with BoxGrinder and thought of creating a simple parallel computing node virtual appliance with some of the software you might need for such an appliance. Here is the appliance definition file (f16-node.appl):

name: f16-node
summary: A Parallel Computing Node appliance based on Fedora 16
version: 1
release: 0
os:
  name: fedora
  version: 16
hardware:
  partitions:
    "/":
      size: 2.0
packages:
  - @core
  - @development-tools
  - gsl
  - openmpi
  - openmpi-devel
  - python-pp
  - scipy
  - ipython
  - python-pip
  - screen

As you can see, this appliance file defines a Fedora Linux based virtual appliance including some of the parallel computing libraries and some other miscellaneous utilities. Now, create a virtual appliance for VirtualBox using: $ boxgrinder-build f16-node.appl -p virtualbox. Once the build process is completed, locate the f16-node.vmdk file under the build/ sub-directory.

Cloning the Virtual Box appliance

Now, that we have our Virtual Box image ready, we shall clone it using the VBoxManage utility’s VBoxManage clonehd command. Create two clones of the f16-node.vmdk file. So you should have three .vmdk files with exactly the same software. Cloning prevents us from the hassle of creating three separate virtual appliances.

Now, create three new virtual machines each using one of the above hard-disks. While setting them up, remember to setupBridged Networking, so that each virtual machine gets a internally (local network) accessible IP address. Check from your host machine if you can ssh to the virtual machines. If yes, you are good to go!

Let me know if that doesn’t work for you.

Related: My earlier post  on creating an ownCloud appliance using BoxGrinder.

As you can see (on opening the screenshot), there is missing header error from ‘/var/www/html/owncloud/apps/files_odfviewer/appinfo/app.php’.  This post here told me that it could be due to a empty line. So that;s it. I went to the file and removed the line no. 8 which was an empty line! And it works fine.

ORIGINAL POST:

In this post, I shall talk about my experiments about creating an ownCloud appliance so that you can have an ownCloud installation running off a virtual machine . This appliance is based on Fedora Linux and I created it using BoxGrinder. To make it visible to the “outside” world, instead of going through the hassle of port-forwarding, I used the awesome PageKite service. Let’s get started:

Understanding ownCloud Installation  (on Fedora)

ownCloud3 installation is simple. On Fedora Linux (Fedora 16), these are the steps you would need to follow: (Soon I think there will be a ownCloud package itself, till then, that is.)

UPDATE: This shell script here does the job for you.

1. Install the following packages: ($ sudo yum install)
   • httpd
   • php
   • php-mysql
   • php-mbstring
   • php-devel
   • mysql
   • mysql-server
   • zlib
   • zlib-devel
   • pcre-devel
   • phpmyadmin
2. Install the php-zip module as outlined here
3. Download ownCloud3 sources from here
4. Extract and copy the resultant owncloud/ directory to /var/www/html
5. ‘cd’ to /var/www/html and set appropriate permission to owncloud directory so that apache can read/write from/to it: $sudo  chown -R apache:apache owncloud/
6. Start Apache server: $ sudo service httpd start
7. Start MySQL server: $ sudo service mysql start and change the root password using $mysqladmin -u root password PASSWORD
8. Create a MySQL database that you want ownCloud to use by going to: http://localhost/phpmyadmin (This step is optional if you want to use SQLite database)
9. Start ownCloud: http://localhost/owncloud and setup other users or simply start using for storing files
10. Use PageKite to make your cloud publicly accessible as detailed here

ownCloud appliance with BoxGrinder

BoxGrinder is a tool to create software appliances for different virtual machine applications and cloud computing providers. It requires an appliance definition file (.appl file) where the base operating system, the software to be installed and other configuration of the appliance is specified.

Next, we shall follow our earlier steps to install ownCloud on Fedora to create an ownCloud appliance based on Fedora with BoxGrinder.  The appliance definition file is here.

Most of it is self-explanatory if you have familiarized yourself with the appliance definition structure. Here I shall talk about the last two sections of the appliance definition file:

files:
 ”/opt”:
    – “http://pecl.php.net/get/zip”
    – “http://owncloud.org/releases/owncloud-3.0.0.tar.bz2″

If you refer to the steps 2 and 3 of the installation steps, you will understand the reason for downloading these two files in /opt.

post:
  base:
    – “cd /opt;/bin/tar -zxvf zip;cd zip-1.10.2;phpize;./configure;make;make install;/bin/echo ‘extension=zip.so’ >> /etc/php.ini”
    – “cd /opt;bunzip2 -d owncloud-3.0.0.tar.bz2; /bin/tar xvf owncloud-3.0.0.tar;cp -r owncloud /var/www/html/;”
    – “cd /var/www/html;chown -R apache:apache owncloud/”
    – “/usr/bin/curl -s https://pagekite.net/pk/ | /bin/bash”

The first three lines carry out the steps 2-5 of the installation instructions. The last line installs PageKite to enable setting up a public access

Next,  we shall build the appliance. We shall build this appliance for use with VirtualBox. Here is the command: 

$ boxgrinder-build -p virtualbox f16-owncloud.appl -d local –delivery-config path:/home/gene/builds

This command will build the appliance and place the gzipped .vmdk file (about 1.4G) in /home/gene/builds. The next step is to create the virtual machine with this as the storage media.

Getting Ready to Serve

Once you have created the virtual machine, boot it and login as the root user (the default password is ‘boxgrinder’. (Enable PAE support from Settings->System->Processor). Then carry out steps 6 and 7 from the installation instruction.

Once that is done, setup PageKite: $pagekit.py –signup and follow the steps. Once the sign up is done, setup your kite using $pagekit.py 80 <yourname>.pagekite.me

Next, carry out steps 8 and 9 from ANYWHERE on the web by replacing localhost by: <yourname>.pagekite.me

ownCloud appliance in VirtualBox

And you have your very own ownCloud instance running from a Virtual machine. I hope to experiment with creating appliances for EC2 and others.

Thank you for reading. Kindly let me know if something didn’t work the way it should, or something could be improved here.

Discussions

I faced a couple of issues while building this appliance. I have posted my query to the BoxGrinder forum here.

]]> http://echorand.me/2012/02/26/owncloud-boxgrinder-pagekite-owncloud-appliance/feed/ 3 Amit owncloud4_error pagekite_vbox