The video is up on YouTube: http://t.co/WorOwbv37w
Slides: https://amitksaha.fedorapeople.org/lca2015/slides.html
Since I could not make it to LCA, Nick Coghlan presented the talk on my behalf. Thanks Nick!
A docker based workflow for working on beaker
While working with beaker‘s code base, I often feel the need to run my tests for a patch/feature and continue to work on with different things while they run, including running other tests testing something different. Currently this is not possible since we start off with a clean database on every test run and simultaneous runs would obviously make one run step on another’s feet.
I finally have an initial docker based prototype for making this possible.
Announcing early access for “Doing Math with Python”
Early Access promotion:
No Starch Press is running a promotion tomorrow (29/01/2015 – PST) that will offer 40% off all Early Access titles on nostarch.com, including Doing Math with Python. The discount code is BRIGHTANDEARLY. Currently two chapters are available and more should be up in the coming days!
I am excited to write that my new book “Doing Math with Python” to be published by No Starch Press is now available via their “Early Access” program – which means if you now buy the book, you will get the chapters as and when they are available and also, the chapters may need more polishing.
The book uses Python 3 exclusively and the Appendix A covers setup and installation instructions for Python 3 and the libraries used in the book. However, that is not yet available. Hence, at this stage, the easiest way would be to use a distribution such as Anaconda on Windows, Linux or Mac OSX (untested, I don’t have access to the OS).
Book Review: Linux System Programming
I received a review copy of the book as part of the blogger review program.
Title: “Linux System Programming (2nd Edition) ” by Robert Love; O’Reilly Media.
Summary:
This book consists of 11 chapters. The first chapter introduces you nicely to the the core topics and lays the foundation for the rest of the book. Files (including some hints on the role of the virtual file system and how they are represented in the Kernel), Input/Output (User buffered I/O, I/O scheduling, Scatter-Gather I/O), Processes (including their creation mechanisms and management), Threads (and how Linux implements them along with a treatment of the POSIX threads library), Memory (Process address space, dynamic memory allocation strategies, and how they work, memory locking) form the core of the book. The second last chapter discusses signal handling. The last chapter of the book is on time (the different types of time, how you can get/set time, measure time elapsed and timers) and is sort of a “standalone” topic for the book. The first appendix discusses the GCC extensions to the C language and can be handy when you read the Kernel source code.
Reactions:
In this book, the author discusses some of the most important topics that one would want to learn about when venturing into the area of “system programming” on Linux. He introduces the topics in a friendly manner adding some fun anecdotes from time to time (what does the “c” in calloc() stand for?).At various places, the reader is given a peek under the hood (for example, pause() is one of the simplest system calls implemented) which can only make the curious reader happy and itchy to download the kernel source code and start grepping. The book includes code examples throughout and hence if you are learning a topic for the first time, these are very useful starting points.
Verdict:
System programming on Linux is an area encompassing number of related topics most of which can fill up whole books on their own. I also could not help comparing this book with “The Linux Programming Interface” by Michael Kerrisk (a book which I own already). Should you buy this book if you already own the latter? Yes, you should. While not being “encyclopedic” and not covering topics such as socket programming at all, Robert Love’s “Linux System Programming” has the right level of treatment and detail for the reader interested in system programming.
Product page:
Fedora 20 Scientific Released
Fedora 20 is now released, which also means the newest release of Fedora Scientific along with other spins are also available.
Download
You can download the spin images from here.
What’s new in Fedora Scientific
The notable additions in this release are:
- Sage, along with Sage notebook.
- SymPy, the Python library for Symbolic mathematics
- The Python 3 versions for scipy, numpy, matplotlib libraries and IPython (including IPython notebook)
- Commons math, a Java library for numerical computing
The Fedora 20 release notes are here.
Fedora Scientific Documentation
I started work on some documentation for Fedora Scientific about a month or so back. It is far from what I want it to be, but you can see the current version here. The first goal that I have in mind is to document all the major scientific tools and libraries that are shipped with Fedora Scientific. By document, I imply links to the official project resources and guides. The second goal is to actually add original content and make it a guide book for Fedora Scientific which may be used as an entry point for Open Source Scientific Computing. Once the guide has taken some shape, an RPM package can be created and distributed with Fedora Scientific so that the entire documentation is available for offline perusal.
Contributing
The Fedora Scientific documentation is an excellent starting point if you are looking to make a contribution to Fedora Scientific. You can view the project here.
If you have some toy/throwaway scripts that makes use of one of the libraries/tools, you may want to contribute it to the
“tests” here. They will help sanity check these libraries and tools during the development of upcoming Fedora Scientific releases.
Discussions and support
Please join the Fedora scitech mailing list.
Suggetions and ideas? Please leave a comment.
C-x C-p for import pdb; pdb.set_trace()
I worked through most of the Learn Emacs Lisp in 15 minute tutorial and learned enough to write something useful:
;; Inset import pdb; pdb.set_trace() on C-x, C-p (defun pdb-set-trace () ;; http://www.emacswiki.org/emacs/InteractiveFunction (interactive) (insert "import pdb; pdb.set_trace()\n")) (global-set-key [(control ?x) (control ?p)] 'pdb-set-trace)
Once you place this function in your Emacs init file, when you press the keys C-x and C-p, import pdb; pdb.set_trace() will magically appear in your Python program (well, anywhere for that matter, since there is no check in place). This is something which I know will be very useful to me. If you don’t like the key combination, you can change it in the last line of the above function.
If you don’t know Emacs lisp, you should try to work through the tutorial. Although, I must say I did make various attempts long time back to learn Common Lisp and Clojure, so it was not so unfamiliar to me.
poweroff, halt, reboot and systemctl
On Fedora (and perhaps other Linux distros using systemd) you will see that the poweroff, reboot and halt commands are all symlinks to systemctl:
> ls -l /sbin/poweroff /sbin/halt /sbin/reboot lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/halt -> ../bin/systemctl lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/poweroff -> ../bin/systemctl lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/reboot -> ../bin/systemctl
So, how does it all work? The answer lies in this code block from systemctl.c:
..
5556 if (program_invocation_short_name) {
5557
5558 if (strstr(program_invocation_short_name, "halt")) {
5559 arg_action = ACTION_HALT;
5560 return halt_parse_argv(argc, argv);
5561 } else if (strstr(program_invocation_short_name, "poweroff")) {
5562 arg_action = ACTION_POWEROFF;
5563 return halt_parse_argv(argc, argv);
5564 } else if (strstr(program_invocation_short_name, "reboot")) {
5565 if (kexec_loaded())
..
program_invocation_short_name
program_invocation_short_name is a variable (GNU extension) which contains the name used to invoke a program. The short indicates that if you call your program as /bin/myprogram, it is set to ‘myprogram’. There is also a program_invocation_name variable consisting of the entire path. Here is a demo:
/*myprogram.c*/
# include <stdio.h>
extern char *program_invocation_short_name;
extern char *program_invocation_name;
int main(int argc, char **argv)
{
printf("%s \n", program_invocation_short_name);
printf("%s \n", program_invocation_name);
return 0;
}
Assume that the executable for the above program is created as myprogram, execute the program from a directory which is one level up from where it resides. For example, in my case, myprogram is in $HOME/work and I am executing it from $HOME:
> ./work/myprogram myprogram ./work/myprogram
You can see the difference between the values of the two variables. Note that any command line arguments passed are not included in any of the variables.
Back to systemctl
Okay, so now we know that when we execute the poweroff command (for example), program_invocation_short_name is set to poweroff and this check matches:
if (strstr(program_invocation_short_name, "poweroff")) ..
and then the actual action of powering down the system takes place. Also note that how the halt_parse_argv function is called with the parameters argc and argv so that when you invoke the poweroff command with a switch such as --help, it is passed appropriately to halt_parse_argv:
5194 static const struct option options[] = {
5195 { "help", no_argument, NULL, ARG_HELP },
..
..
5218 case ARG_HELP:
5219 return halt_help();
Fooling around
Considering that systemctl uses strstr to match the command it was invoked as, it allows for some fooling around. Create a symlink mypoweroff to /bin/systemctl and then execute it as follows:
> ln -s /bin/systemctl mypoweroff
> ./mypoweroff --help
mypoweroff [OPTIONS...]
Power off the system.
--help Show this help
--halt Halt the machine
-p --poweroff Switch off the machine
--reboot Reboot the machine
-f --force Force immediate halt/power-off/reboot
-w --wtmp-only Don't halt/power-off/reboot, just write wtmp record
-d --no-wtmp Don't write wtmp record
--no-wall Don't send wall message before halt/power-off/reboot
This symlink is for all purpose going to act like the poweroff command since systemctl basically checks whether ‘poweroff’ is a substring of the invoked command.
To learn more, see systemctl.c
Related
Few months back, I had demoed invoking a similar behaviour in programs where a program behaves differently based on how you invoke it using argv[0] here. I didn’t know of the GNU extensions back then.
