Raspberry Pi: Personal Perspectives

I have been playing around with the Raspberry Pi (Pi) for two months now. I started by installing Raspbian on a spare 4 GB memory card. I also had a wireless Wifi dongle and a wireless keyboard-mouse lying around. I plugged the WiFi dongle and the dongle for the wireless keyboard-mouse in the two USB ports of the Pi and when I plugged in my spare mobile charger into the Pi’s power input, the Pi booted. I felt lucky to see the display on the TV without any hassles. So far all in all, out of the box experience and all I had bought was the Pi. I already had the other stuff lying around. It was fun to see something on TV that I am used to seeing on a computer monitor.. The next thing I did was fire up the Python interpreter and tweeting the output of os.uname( ).

The Board by Amit Saha

Next things I tried to do was like what I had seen most people do was trying to make a media centre out of the Pi. And few other things like setting up a audio server, etc came to my mind. But then, I thought to myself – I don’t have a huge media collection, nor do I play games, nor do I sit back a lot and watch stuff. I think I am better off doing something else with the Pi. Its Linux in a more constrained environment after all. I fiddled around a bit with the Pi after that. Then, finally couple of weeks back, I installed the Fedora 17 ARM remix on the Pi. That was the point where I really started using the Pi.

Fedora 17 on the Pi

One of the first things I liked about the Fedora remix was that the SSH server was installed and enabled out of the box. Once I was done with firstboot, I plugged the TV jack out and work on the Pi by SSH-ing into it (The Pi is connected to my home network via the Ethernet port).

I have been documenting my experiments with Fedora on the Pi here [1]. As you can see, most of the programming languages, frameworks and tools there is all the typical things you would be using on any Linux system – BASH, Python, Ruby, Flask – you name it. It was like – so it worked on Linux on the big machine, let’s say if its there on Pi and everything was pretty much so close to same that it started to seem like a chore – more about documenting what I was doing in the hope that it might be useful to some one and may be I can use them as a base for a book/article in the future. There in lies the essence of the awesome work done by the Fedora ARM team which made things a chore. Thank You folks! And of course to all the Python programmers who have made their packages available via PyPi. pip-python was indispensable.

No more a chore

Things changed a bit yesterday. I fiddled around with the GPIO pins for the first time. Using the WiringPi’s gpio command, it was as easy as it gets. I then used the Python package RPi to interact with the Pi pins from Python and it was quite dandy. As you can see from the documentation [2], among the two packages one of them was already installed and the other was installed from PyPi. Finally, that was something I can’t do so easily on my big Linux machines. That also meant, I do not have to use Arduino to fiddle around with some basic, hobbyist electronic stuff. In one of the coming days, I might hook up the Arduino with the Pi.

Role of Abstraction Layers

The most basic operating system course introduces it as an interface between the user and the hardware – an abstraction separating software from the hardware. I think the Pi is a great example for educators to show exactly what they mean. Have a traditional computer and a Pi side by side – write the same program and run it both the devices show them. That should be quite intuitive. The fact that I started finding things a chore was because the operating system was abstracting out the hardware from me.

Experimenting with C

The good thing (and the bad thing) about Python and Ruby (for example) is that you are one layer higher (than C) in the abstraction hierarchy. C is one layer close to the bare metal and hence should be an educational exercise to play with it on the Pi and compare it with an an Intel or AMD computer. That should help illustrating the differences you need to ponder about when you are programming in C (or C++) and intend your program to be also working on other architectures.

I touched on this in this article titled “Compilation and Interpretation in C and CPython“.

Conclusion

The experiments will continue.

Links

Mining the Fedora Infrastructure Bus

If you listen to the Fedora Infrastructure Bus, you will hear things like:

fedora-infrastructure:org.fedoraproject.prod.fas.user.create
{u'certificate':
u'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVSVENDQTY2Z0F3SUJ
BZ0lCSmpBTkJna3Fo\na2lHOXcwQkFRVUZBRENCb0RFTE1Ba0dBMVV
FQmhNQ1ZWTXgKQ3pBSkJnTlZCQWdUQWs1RE1SQXdE\nZ1lEVlFRSEV3ZFNZV3hs
YVdkb01SY3dGUVlEVlFRS0V3NUdaV1J2Y21FZwpVSEp2YW1WamRE
RVBN\nQTBHQTFVRUN4TUdabVZrYlhObk1ROHdEUVlEVlFRREV3Wm1aV1J0YzJjeER6QU5
CZ05WCkJDa1RC\nbVpsWkcxelp6RW1NQ1FHQ1NxR1NJYjNEU
UVKQVJZWFlXUnRhVzVBWm1Wa2IzSmhjSEp2YW1WamRD\nNXYKY21jd0hoY05NVEl3TnpFMU1
qSXdNRFE1V2hjTk1qSXdOekV6TWpJd01EUTVXakNCMURFT,
u'i': 1, u'timestamp': 1353031870.694725, u'topic':
u'org.fedoraproject.prod.fas.user.create', u'signature':
u'AvVPjUn0qP+W9qYLDkPFDBdmQDq66NJd2TuPfZZYYbwEkCSOmKsbNnGyGyVhM0n4SoqlNWR0KG49
\nz3FSBgHO2Tytnj+ZltMmeg8GNwaf5ys+bAaiL81AOTeG
ALtPD/iYGiXB0vIjZ685IdPMlu9ZZZej\nauUcUnKuacpzcy17/NA=\n',
u'msg': {u'user': {u'username': u'odemia'}, u'agent': {u'username':
u'odemia'}}}

which in plain text means that a new FAS username was created which goes by the name ‘odemia’.

Or, for example:

fedora-infrastructure:org.fedoraproject.prod.bodhi.update.comment
{u'certificate': u'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVTVENDQT
dLZ0F3SUJBZ0lCRnpBTkJna3Fo\na2lHOXcwQkFRVUZBRENCb0RFTE1Ba0dBMVVFQmhNQ1
ZWTXgKQ3pBSkJnTlZCQWdUQWs1RE1SQXdE\nZ1lEVlFRSEV3ZFNZV3hsYVdkb01SY3dGUVl
EVlFRS0V3NUdaV1J2Y21FZwpVSEp2YW1WamRERVBN\nQTBHQTFVRUN4TUdabVZrYlhObk1RO
HdEUVlEVlFRREV3Wm1aV1J0YzJjeER6QU5CZ05WCkJDa1RC\nbVpsWkcxelp6RW1NQ1FHQ1Nx
R1NJYjNEUUVKQVJZWFlXUnRhVzVBWm1Wa2IzSmhjSEp2YW1WamRD\nNXYKY21jd0hoY05NVEl
1Evcit2TFFZZWoxN1cwCi0tLS0tRU5EIENFUlRJRklDQVRF\nLS0tLS0K\n', u'i': 1, 
u'timestamp': 1353038721.092251, 
u'topic': u'org.fedoraproject.prod.bodhi.update.comment', u'signature': 
u'kfsHMnH0zHp6t11woX9uH34t3qDGOtzRHHgrIuUMaBlB8XbNfXl2lQOzhR5iQd3iUIMUh71Q0UPL\njjaIvRJWxL/yz75oiq
btBMYUabi3X0mOmz3dj0rzGeIJVrwVdz+u5ePeBhR6rTDAvrmknrw6lgnr\nyVG7eFk6mQUGXk+MM9o=\n', u'msg':
{u'comment': {u'group': u'proventesters', u'author': u'adamwill', u'text': u'mostly working okay here', 
u'karma': 1, u'anonymous': False, u'timestamp': 1353038719.0, u'update_title': u'anaconda-18.29-1.fc18'}, 
u'agent': u'adamwill'}}

which corresponds to adamwill’s comment on bodhi [1].

If you start storing these messages emitted, you can mine for some interesting statistics. (I am storing all the messages in a file ‘messages’ using a particular format. See the Python source code at the end)

For example, in the last 20 hours or so, 16 new users registered with FAS:

$ cat messages | grep 'fedora-infrastructure:org.fedoraproject.prod.fas.user.create' | wc -l
16

And 175 edits was made to the various wiki pages:

$ cat messages | grep fedora-infrastructure:org.fedoraproject.prod.wiki.article.edit | wc -l
175

You can also look for rawhide and branch compose tree messages:

$ cat messages | grep fedora-infrastructure:org.fedoraproject.prod.compose
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.mash.start
fedora-infrastructure:org.fedoraproject.prod.compose.branched.start
fedora-infrastructure:org.fedoraproject.prod.compose.branched.mash.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.mash.complete
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.rsync.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.rsync.complete
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.complete

And as you can imagine, the possibilities are well, endless. Check out the documentation here at [2]. The consumer script I used is at [3]. Thanks to threebean for his help.

[1] https://admin.fedoraproject.org/updates/FEDORA-2012-18105/anaconda-18.29-1.fc18?_csrf_token=482cef87c4c42a4f60ba3faf97bbe16acad4e935
[2] Fedmsg Documentation
[3] Naive Consumer Script

Conflicting libdb dependencies: rpm-libs + httpd 2.2

I ran into a segfault-ing web application while attempting to get Beaker server working on Fedora 17. Beaker uses TurboGears for the web application using mod_wsgi to talk to Apache (httpd-2.2). (I am a little new to the web application world, so ignore any misuse of terms).

The particular use case that triggered the crash was uploading a task RPM to the Beaker server. That was some hint into where to look for the faulting code. The task RPM was being saved fine on the disk, so there was no problem in writing (permissions, etc). So, there had to be a problem in reading the RPM file (which is what Beaker does to retrieve the files and task specifications).

To investigate whether it was a problem reading the RPM file, I simply copied the relevant code from Beaker sources to a standalone Python script. No problems there. Hence, there had to be something wrong with the combination of using the rpm libraries (rpm-libs) together with the web application (httpd).

Debugging httpd with gdb

After futile efforts of inserting exception handling codes combined with logging in the code to bracket the code which was causing the crash, I decided to bite the bullet and just go the gdb way. Turns out it was simpler than I thought. Run ‘httpd’ in single process mode, and then monitor the error_log file for the wsgi daemon process ID and then attach to it using gdb in another terminal.

Start httpd in single process mode:

# gdb /usr/sbin/httpd
(gdb) run -X

Then attach gdb to the wsgi daemon process (You should ensure via your wsgi configuration to create only one mod_wsgi daemon process with a single thread):

# gdb /usr/sbin/httpd attach 
(gdb) cont.

Once I had done this, I simply performed the action that triggered the crash, and I had a good stack trace, first few lines of which were:

#0 0x0000000100000001 in ?? ()
#1 0x00007fffd7fe1097 in db_init (dbhome=0x7fffdf7a9e70 "/var/lib/rpm", rdb=0x7fffdd052970)
at backend/db3.c:151
#2 dbiOpen (rdb=rdb@entry=0x7fffdd052970, rpmtag=rpmtag@entry=0, dbip=dbip@entry=0x7fffe399ff38,
flags=flags@entry=0) at backend/db3.c:551
#3 0x00007fffd7fe8e53 in rpmdbOpenIndex (db=db@entry=0x7fffdd052970, rpmtag=rpmtag@entry=0, flags=0)
at rpmdb.c:149
#4 0x00007fffd7fe93ef in openDatabase (prefix=, dbpath=dbpath@entry=0x0, dbp=dbp@entry=
0x7fffdf797648, mode=mode@entry=0, perms=perms@entry=420, 
.
.
.

So that pretty much confirmed that it was something that httpd did not like about the rpm-libs which caused it to crash the application. Discussing with Dan, this was indeed a case of conflicting shared libraries and bit more of looking around we found that this was the Berkeley DB database library (libdb) that was the culprit. httpd had both libdb-4.8 and libdb-5.2 loaded in its process maps, also verified with ‘lsof‘ (Thanks to StackOverflow for the lsof tip):

# lsof /lib64/libdb-4.8.so

COMMAND   PID   USER  FD   TYPE DEVICE SIZE/OFF    NODE NAME
httpd    9722 apache mem    REG  253,1  1555128 1189531 /usr/lib64/libdb-4.8.so
httpd   10578 apache mem    REG  253,1  1555128 1189531 /usr/lib64/libdb-4.8.so
httpd   18828 apache mem    REG  253,1  1555128 1189531 /usr/lib64/libdb-4.8.so
httpd   18832 apache mem    REG  253,1  1555128 1189531 /usr/lib64/libdb-4.8.so
gdb     18863   root mem    REG  253,1  1555128 1189531 /usr/lib64/libdb-4.8.so

[root@asaha temp]# lsof /lib64/libdb-5.2.so 
COMMAND    PID   USER  FD   TYPE DEVICE SIZE/OFF    NODE NAME
gdb      18824   root mem    REG  253,1  1756808 1204315 /usr/lib64/libdb-5.2.so
httpd    18832 apache mem    REG  253,1  1756808 1204315 /usr/lib64/libdb-5.2.so
gdb      18863   root mem    REG  253,1  1756808 1204315 /usr/lib64/libdb-5.2.so
qemu-kvm 30866   qemu mem    REG  253,1  1756808 1204315 /usr/lib64/libdb-5.2.so

As you can see, both the libdb versions is mapped to httpd’s process space. Since, httpd itself depends only on libdb-4.8, there is something else in the web application which is bringing in libdb-5.2. That something turned out to be rpm-libs:

# yum deplist rpm-libs |  grep libdb
  dependency: libdb-5.2.so
   provider: libdb.i686 5.2.36-5.fc17

So, that’s the problem and the reason for the crash.

Solution

There is no solution on Fedora 17 at this point of time to this other than trying to get httpd-2.4 which is linked against libdb-5.2.so. However, the Fedora 18 release, currently in development ships with httpd-2.4 which is linked against libdb-5.3.so, same as the rpm-libs version it is shipped with. And indeed, the above crash did not occur there.

Reproducing

To reproduce this with a minimal application, I taught myself how to integrate Flask with mod_wsgi and wrote this simple Flask application, which you can check out here. Follow the steps on the Flask docs for help.

Resources

Thanks to Dan on #fedora-qa and Dan in real life who helped me with debugging the crash and thanks to Graham Dumpleton for the mod_wsgi help on #pocoo. Here are some of the docs I referred to:

Related bug reports

httpd+mod_perl and BerkeleyDB incompatibility

GSoC 2012: The Experience

The evaluations are in and I have formally passed the Google Summer of Code, 2012 as part of the Fedora project. It was a huge learning experience. I was introduced to a number of new tools and techniques which I hadn’t heard of before. I do not yet know how to literally “handcraft” Fedora ISO’s, however I at least know the tools for the job and have got familiar with working with Koji, the various repositories and the learnt about the “lifecycle” of a package. In the process I hope my project will be useful to the Fedora QA, which is where we will be working towards.

Takeaways

This was my first major open source project with a deadline to complete. And since I used a number of third party Open Source software, when I was stuck with a problem, the only avenue for help I had was the mailing list/project lead. That had its own shortcomings. I think there is an important lesson to learn. If you are using Open Source software in Business, you better buy support. In the same vein, if you are a Open Source software expecting to be used in anything serious, provide commercial support.
I had multiple priorities to deal with during the past few months – professional and personal and I wanted to do them all equally well, because all were dear to me. I am glad I managed everything pretty well in the end – Just decide what you want to do, and work towards it. Things have a strange way of coming together at the end and leave you smiling.

Thanks

Any successful endeavour is usually not achieved alone. First and foremost, thanks to Tim Flink – my project mentor for seeing this through. Thankfully, my proposed idea of a web-based build service resonated with something which he had in mind and had already worked on for a bit and that was the beginning of it all. He also had to write extra long emails explaining to me what exactly we would be attempting to achieve. Thanks, Tim for your co-operation and guidance.

Thanks are due to the community of the following projects: celery (Thanks Ask Solem and others), Flask (Thanks guys for making me believe even I can write web applications), Fabric, Zdaemon, Read-the-docs, Sphinx and of course the Fedora community in various ways. Python made working on the project a delight. Finally, thanks Google for organizing the Summer of Code. It definitely helps in pooling a dedicated task force working on Open Source projects.

And all the non-technical factors that makes a person take up such endeavours in the first place – when that little push is needed there is always someone behind the scene giving that push- explicitly or implicitly. Thanks to Protyusha for that.

And last but not the least, sometimes you need people against you to always spur you on. So, thanks to those people who knows only how to try to pull someone down.

For those of you kids who are thinking – hey this guy is writing like he has scaled Everest, I would like to tell you: Learn to celebrate the smallest of things, it just makes it that more precious.

GSoC 2012: On-Demand Fedora Build Service: Final Update

The Google Summer of Code’12 working deadline was yesterday. So that brings me to this last project update. In a last addition, I added Fedora Account System (FAS) authentication to the Web application. This was really easy, thanks to the Flask-FAS plugin. The latest documentation can be referred to here and gives you a fair idea of how you could setup and use this service.

However, this is just the end of the Google Summer of Code. The original target of this project was to make it useful to Fedora QA and that’s what we will be working towards next. So, feel free to read the docs and try it out and file issues.

GSoC 2012: On-Demand Fedora Build Service: Update #11

Its been a while since I posted an update, and this will probably be the last before the final update as part of the Google Summer of Code.

The big update is that I have got pretty decent documentation up now. You may browse it here [1]. Its up to date with the current code base [2]. Besides the user documentation, it also contains some of the internal details about the tools/libraries/design decisions that have been taken in the project.

If you have got some free cycles to spare (human + computer), you may want to give it a shot to try building images for yourself. As a tip, the local mode, involves least amount of setup and is easiest to try out what is in there. If you see anything broken, feel free to file an issue.

As an aside, I was literally blown away by the awesomeness of Sphinx and Read the docs. Really great tools and services.

Links:

[1] Documentation: http://on-demand-fedora-build-service.readthedocs.org/en/latest/

[2] GitHub: https://github.com/amitsaha/gsoc2012_fbs

GSoC 2012: On-Demand Fedora Build Service: Update #10

In my latest commit [1], I added a number of features to the code which I had in mind:

Application wide Logging
Basic Build Monitoring
Email notification

These features are most useful when the service is deployed with multiple worker nodes. For local image building, these are not so important. I also added support for specifying local filesystem as staging – useful for local image building.

Next up, I have a few things to work on. Test the Web UI and Rest API interface again. Update the HOWTO (its outdated already!). And few other things remain.

Demo of Email Notification

When you submit a build request (in multiple worker nodes mode), you get an email upon notification:

Your Image Building Request have been submitted. You may monitor the progress by going to http://192.168.1.4:5100/log/tmp/imagebuild_134292350407.log. You will also recieve an email upon completion.

And then when the job completes, another email:

The build was completed by worker:: 192.168.1.4. Detailed log:

2012-07-22 12:18:24,074 – Registered a new Image Build request fromamitsaha.in@gmail.com
2012-07-22 12:18:24,075 – Image type:: dvd
2012-07-22 12:18:29,074 – Image Build notification sent
2012-07-22 12:18:29,383 – Starting the Image Build Process
2012-07-22 12:18:29,385 – DVD image arch requested should be the same as the build arch
2012-07-22 12:18:29,386 – Image building process complete
2012-07-22 12:18:29,387 – Error creating image. Transferring Logs.
2012-07-22 12:18:29,388 – Initiating local transfer of logs to /tmp/staging
2012-07-22 12:18:29,401 – Logs available at file:///tmp/staging

At this point of time, the best place to understand what is happening is to begin with the command line clients in cli/ : build_cli_basic.py and build_cli.py.I Should have an updated HOWTO soon.

Links:

[1] Latest commit: https://github.com/amitsaha/gsoc2012_fbs/commit/cf23dcc17d38924296eb099f6d1c2a440efb3d82

GSoC 2012: On-Demand Fedora Build Service: Update #9

The code saw a few additions/enhancements since the last update:

There is now a command line client directly submitting build jobs and another client using the basic REST interface exposed via the Web application [1].
Remote Kickstart files specified via a http:// or ftp:// are now supported. Once again, this has to be ‘ksflattened’.
I added some validation on the web based interface. However it happens on the server side using Flask’s pre_validate( ) method. Ideally I would like to familiarize myself with enough Java Script to do this on the client side
I started writing some Unit tests using py.test and Mock. But, I am currently waiting for some guidance from Tim to really write some good tests.

With the Mid-term evaluation due in less than a week, I am quite happy with the way things are progressing at this moment. The TODO and GOOD-TO-HAVE notes down the things I would like to finish before the project deadline is over [2].

Links

GSoC 2012: On-Demand Fedora Build Service: Update #8

I spent some time cleaning up the code, mainly using pylint as an indicator. Also rewrote the fabfile to use fabric commands rather than native Linux commands wherever possible and is cleaner now. The HOWTO is also updated now.

This code refactoring has brought home (quite strongly!) the need for having my unit tests up and running soon. I plan to use py.test for my testing, and hopefully will have some tests ready in a week.

The code is available at https://github.com/amitsaha/gsoc2012_fbs

GSoC 2012: On-Demand Fedora Build Service: Update #7

In my last update, I reported that I had a functional build service, capable of harnessing multiple build nodes and a functional Web UI as well. The process to deploy the build service was however quite manual and tedious. I was looking for a way to help do this without as much manual intervention as possible and fabric (Thanks, Tim) was the answer. I wrote a fabfile for the whole task, called ‘deploy.py’. As outlined in my earlier post, I am using celery to distribute the build tasks, so the celeryd process needs to be running on the build daemons. I found zdaemon (Thanks Jan on fabric mailing list) to be the easiest way to run the celeryd process as daemons. The updated code is available in the repository now [1].

There is a HOWTO [2] document, which should help you deploy the service and run your own home based build service. I recently used it to build myself a Fedora Scientific ISO. Please don’t abuse it. There is hardly any error handling now. I am however accepting bug reports now. Thanks for all your help.

[1] https://github.com/amitsaha/gsoc2012_fbs/
[2] https://github.com/amitsaha/gsoc2012_fbs/blob/master/HOWTO

Bits from /home/gene

Category Archives: Fedora