What I did is simple: I published the square_func using PiCloud’s documentation. Then I used libcurl’s C Interface to replicate the curl statements provided in the documentation. I have
written the client in two parts: client1.c and client2.c. client1.c invokes the published function, hence gets the Job ID. client2.c then uses this jobID to get the result. Here are the source codes:

/*client1.c*/
/* This part of the client invokes the REST API of
   PiCloud and retrieves the Job ID
   http://docs.picloud.com/rest.html#invoking-functions
*/

#include <stdio.h>
#include <curl/curl.h>

int main(void)
{
  CURL *curl;

  /* Make sure you set this appropriately*/
  char *url="https://api.picloud.com/r/3222/square_func/";
  CURLcode res;
  
  curl = curl_easy_init();
  if(curl) {
    /* First set the URL that is about to receive our POST. This URL can
       just as well be a https:// URL if that is what should receive the
       data. */
    curl_easy_setopt(curl, CURLOPT_URL, url);
    /* Specify the user/pass */
    curl_easy_setopt(curl,CURLOPT_USERPWD,"apikey:secretkey");
    
    /* Now specify the POST data */
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "x=5");
    
    /* For HTTPS */
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);

    /* Perform the request, res will get the return code */
    res = curl_easy_perform(curl);
    printf("\nResult of Operation:: %d\n", res);

    /* always cleanup */
    curl_easy_cleanup(curl);
  }
  return 0;
}

/*client2.c*/
/* This part of the client retrieves the result given the
   job ID as the argument*/
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>

int main(int argc, char *argv[])
{
  CURL *curl;
  char url[80];
  CURLcode res;

  if (argc==1)
    {
      printf("Usage: ./client2 <jid>\n");
      exit(0);
    }

  
  strcpy(url,"https://api.picloud.com/job/result/?jid=");
  strcat(url,argv[1]);

  

  curl = curl_easy_init();
  if(curl) {
    /* First set the URL that is about to receive our POST. This URL can
       just as well be a https:// URL if that is what should receive the
       data. */
    curl_easy_setopt(curl, CURLOPT_URL, url);
    
    /* Specify the user/pass */
    curl_easy_setopt(curl,CURLOPT_USERPWD,"apikey:secretkey");
    
    /* for HTTPS */
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
    
    /* Perform the request, res will get the return code */
    res = curl_easy_perform(curl);
    printf("\nResult of Operation:: %d\n", res);

    /* always cleanup */
    curl_easy_cleanup(curl);
  }
  return 0;
}

Working

[gene@zion tmp]$ ./client1
{“jid”: 57, “version”: “0.1″}
Result of Operation:: 0

[gene@zion tmp]$ ./client2 57
{“version”: “0.1″, “result”: 25}
Result of Operation:: 0

Hope it works for you. PiCloud is fun!

For an Evolutionary Algorithm, there are a number of ways it can be parallelized. One of the easiest things to do is to run parallel instances of the algorithm with different initial random seeds. (This is a common practice in the research community – since starting with different random seeds allows testing of the robustness of a new algorithm or the correct  implementation of an existing one). The first exercise that I tried with Pyevolve + PiCloud is to run Pyevolve’s Genetic Algorithm implementation with 10 different initial seeds on PiCloud  – all running in parallel. Here’s how, you can go about it:

Installing Pyevolve

1. Clone the git repository from https://github.com/perone/Pyevolve
2. Install Pyevolve: $ sudo python setup.py install
3. Just to check if it has been installed properly, just run one of the examples in the examples/directory

Setting up Picloud

1. To setup PiCloud, please follow the instructions at http://www.picloud.com/ starting with registering for a free account
2. Once you have installed the PiCloud client, please verify the installation by typing ‘import cloud’ from your Python prompt and see that it doesn’t throw up any errors

Getting Ready to Launch

For the purpose of this experiment, I shall be using one of the examples that ship with Pyevolve: pyevolve_ex7_rastrigin.py after some modifications. The modified file is here. Specifically, we change the main() function to run_ga() taking two parameters: seed and runid. The seed parameter is used to provide a random initial seed (used when creating the GA engine using ga=GSimpleGA.GSimpleGA(genome,seed)). The CSV adapater is initialized to store the statistics in a CSV file, where the runid file is used for specifying the name of the file.

Okay, now I shall describe the main driver script which I used to run this GA in the cloud. Here it is:

#!/usr/bin/python
# Simple demo to show how to run the Pyevolve
# Evolutionary Algorithms framework on PiCloud
# Pyevolve: http://sourceforge.net/projects/pyevolve/
# PiCloud: https://www.picloud.com/

# Amit Saha
# http://echorand.me

from pyevolve_rastrigin import *
import cloud

cloud.setkey(<API KEY>, 'SECRET KEY')

# List of Random seeds and run-Ids
# assuming 10 runs
seed_list=[100*(i+1) for i in range(10)]
runid_list=[i+1 for i in range(10)]

# calls the method defined in pyevolve_rastrigin.py
# which initiates the GA execution.
# Execute the code on PiCloud
jids = cloud.map(run_ga,seed_list,runid_list)

# check if the jobs are complete, if yes
# pull the stat files
cloud.join(jids)
for i in range(10):
    cloud.files.get('stats_' + str(i+1) + '.csv','stats_' + str(i+1)+'.csv')

One of the first things you need to do is set the API Key and Secret key that you will be using to access PiCloud. The API Key and the Secret key can be seen in the “API Keys” page of your PiCloud account. This is done by the line cloud.setkey(, 'SECRET KEY'). Next, we use the cloud.map() function to call the run_ga function for each pair of the values of the lists seed_list and runids_list. This is a efficient way of running the same function in the cloud, but with a different set of parameters. Once that is launched, you can see the state of your jobs from your PiCloud’s account’s Jobs page.

Next, what we do is we wait for all the jobs to finish using cloud.join(jids) and then pull all the statistic CSV files from PiCloud’s storage to the local file syste using cloud.files.get('stats_' + str(i+1) + '.csv','stats_' + str(i+1)+'.csv') (More information on using the cloud.files module is here). Once the files have been copied, you can see the results from the CSV files – each CSV file representing the result of one run of the GA. We have not yet talked about how the CSV files were created, however.

Creating files on PiCloud from Pyevolve

The source code of CSV adapater, where the CSV files are created is in pyevolve/DBAdapters.py. The open() method of class DBFileCSV is where the file is opened for writing and the close() method of the same class closes the file handle when the writing is finished. However, this method of creating files won’t work for PiCloud – rather the data won’t be retrievable for our client. We have to create the file the PiCloud way – which is to use the cloud.files.put() function. All I did here was add the line cloud.files.put(self.filename) in the close() method. And I reinstalled Pyevolve module and it all worked fine. You may find my modified DBAdapaters.py file here.

Conclusion

I hope to discuss with the Pyevolve folks what they think of my changes to the DBAdapter class and see if they suggest a better way. Like I mentioned in the beginning, this is a very naive way of harnessing the power of parallel computing in Evolutionary Algorithms. I hope to explore more in this direction with PiCloud. If you have any queries, please leave a comment.

Thank you for reading.

Arggh is her oft-spoken word,
Speaking to an audience her forte,
Impressing others is a default,
Proving others wrong is her habit.

No, She is not your average lady,
She is the lady.

She dresses up in half pants, three quarters and full pants,
She wears boleros and salwars,
She carries them all with grace and a certain beauty,
Afghanistan interests her. So does pottery,
Photography interests her, so does quantitative finance,
Painting interests her, so does the middle east.

No, She is not your average lady.
She is the lady.

She wakes up nights studying,
She loves listening to music in the dead of the night,
She loves cleaning the house even when she is sick.

No, She is not your average lady,
She is the lady.

She proposes ideas for books,
She reviews photographs and blog posts,
Her taste’s rich, Her thoughts vast,
Her dreams big,
Her appreciation for life much.

No, She is not your average lady.
She is the lady.

She cries, she laughs.
She fights, she hugs.
She abuses, She praises.

No, She is not your average lady.
She is the lady.

No she is not of a mild nature.
No she is not very easily understandable.
No it’s not always easy to understand how to make her happy.

No. No. No.
She is not your average lady.
She is the lady.

Here I shall summarize some of my own findings and the current state:

The Current State

MIT App Inventor Status Update: appinventoredu.mit.edu/developers-blo… #android

— Amit Saha (@echorand) January 21, 2012

The current state is in transit. For a more official status update, please see the above link. The summary is that the source code for the App Engine was released a day before. One of the things it allows you to do is run your App Engine service locally. Please see this link for the instructions. Note that you would need ~4G of RAM to give Java enough memory to play around. If you want to host the open source version of the service on Google App Engine, please follow my forum posting here. If you want to continue using the MIT published JAR’s for running your own App Inventor instance, please follow the link here. Basically, to host your service, there are two things you must do (detailed here)

1. First, upload the App Inventor application to Google App Engine
2. Setup your build server – this needs to be a publicly accessible machine. By that I mean, this IP should be visible to the world. People have used Amazon EC2. I just used a web host/domain I own personally.

Using the Service

If you just want to start using the service without going into the hassles for setting up yourself, sign up for MIT’s own testing service here. You could also send me an email or comment to be added to the whitelist of users for my own service at http://appinventordemo.appspot.com and just in case you just want to have a look at what all is you may check out the references and tutorials here at http://echorand.com/appinventor/about/.

Getting Help and Sharing your Findings

• Working with the Open Sourced App Inventor codes: Google Group – App Inventor Open Source Development
• Hosting the MIT JARS: Google Group – Google Group -mit-appinventor-jars
• Forums for Users and Educators: http://echorand.com/appinventor/forum/

Miscellaneous

• Some issue I faced while installing the generated APK’s on the Android device:

APK signing issue finally solved. JDK downgrade needed. Thread: groups.google.com/group/mit-appi… Thanks Hal #android

— Amit Saha (@echorand) January 22, 2012

• App Inventor for All Project: http://ai4a.jsoft.com/

• Resources to Learn about App Inventor: http://echorand.com/appinventor/learn/

• Once you have got a place to try out App Inventor, remember the only condition is : you agree to have fun! Let us know on one of the forums or in a comment.

Faces are Us (Source: Clicked by your's truly)

“All the world’s a stage,
And all the men and women merely players:
They have their exits and their entrances;
And one man in his time plays many parts”

So began Shakespeare’s Monologue in As You Like It. I think the human race has taken him a little too seriously and try really hard everyday to prove the relevance of Shakespeare in our daily lives. To express myself more accurately, I would rewrite these four lines as:

“All the world’s a stage,
And all the men and women merely actors:
They have their exits and their entrances;
And one man in his time puts on many masks”

Look back a little in your life – may be as little as even 12 months in your life and you will know what I mean. If you don’t, lucky you. I am keeping my fingers crossed. For, its the many faces of man and woman that can ruin you. You might befriend “friend”, but hey, “friend” is not actually what you think “friend” is. “friend” has an agenda, a reason to be with you – and no it isn’t because “friend” likes you or admires you. “friend” just wants to have some fun, at the cost of your faith, trust and emotions. Its funny how this “friend” reacts to situations in your life. Here are some instances:

• “friend” is a great friend to you when you are sad, lonely, perhaps you miss someone very important in your life, or you are in a mental turmoil for something. You think “friend” is a great friend, so you befriend “friend” and really make “friend” important in your life thinking that “friend” wants nothing but your good. Then you find some happiness in your life, and “friend” is NOT happy. “friend” doesn’t even express happiness.
• “friend” is your friend for many years. “friend” even goes to the extent of telling you to find your own happiness and go for it. You go and find it, but “friend” isn’t happy anymore for you. Another “friend”, another Face. Did “friend” not expect that you would actually go and get your happiness?
• You have loved “friend” dearly, (a lot like brotherly love, if you please) you do a lot for “friend”, buy her books, making sure “friend” gets all  that’s needed. You ask “friend” for something really small in return, and “friend” doesn’t feel that its necessary. “friend” doesn’t care anymore of your existence.
• …and it goes on

So, what is the problem with our “friend” here? Is “friend” jealous, angry, sad, disappointed with you? Or is “friend” just pretending to be something which actually “friend” is not. Or, is it just that you are wrong? That you couldn’t see what it was actually there. May be you are wrong, but from personal experience, you can’t be wrong all the times. You are being playing into the hands of a bunch of moral-less individuals who don’t even know what friendship, love or being human is. Being happy for someone else is a far cry from them. If you analyse “friend”, you will see that “friend” doesn’t even know what to make of this life- and hence try to be different at different points of time. They lack the appreciation for beauty – they lack the good will to come and pat your back and say – “Hey, good job! Keep it up.”, they lack the will power or force to chase their dreams, they don’t slog for their career or for their life. They don’t know how to be a friend or a lover.

Look around, do you see a “friend”? If yes, avoid “friend”.

And Dear “friend”, please put on a real mask, so that we know whether you are a batman or a spider-man. Enjoy your days in the sun because people like you are usually outlived, if not outnumbered.

PS: Excuse the liberal use of “friend”. It was intentional to highlight the farce that some people make friendship to be.

The concept of life is thus quite open to well, conceptualization. We give it a different meaning or rather it means different to us at different points of time. Its rather easy to wander around in life in the name of exploring, flexibility or being adventurous. And it is during this wandering stage of life that big mistakes are made. Life is normally a long period of time we are dealing with. And its all the more important to respect this fact and act with the fear of consequences in mind. Short sightedness and a closed room thought process results box like thinking. And that is when the big picture is missed. The term big picture may sound like a cliche term. But it actually is important to  see it. When we see the big picture, we tend to concentrate on our insignificance as a human and as a.species as compared to the vastness that surrounds us. And the day we realize our own insignificance and ignorance, we start growing sensible, we start acting with the big picture in mind. But the big question is how to not lose your way? how to not miss the big picture?

As individuals we tend to follow trajectory in our lives. There is a beginning and there is an end. We rise and we fall. During this rise and fall, its easy to forget where you started from. Its easy to forget your roots. Root could be just yourself, a person you love or trust most or it could be a set of principles you live by. Being rooted is important to stop wandering from. The concept of being rooted is similar to the concept of being anchored to something or someone which helps you from being lost in the open sea of life. In earlier times and probably even today, in a section of Indian society – marriage is seen as a way of making the bad son from going berserk. The responsibility of a wife is believed to be sobering down this son. May be it works. But again the principle is the same – being anchored or rooted is important.

Sometimes, you might have wandered so much that too much has been damaged – there is no point of return – or so you think. But its never too late to return to your roots, to your anchor and start fixing things. The struggle must go on. You must keep fighting. For it is in our mistakes that we learn never to commit them. There won’t be another life. So, don’t go to the GAP yet.

The show must go on.

Its funny how easy it is to find yourself aloof from what’s happening around you. The clutter of voices suddenly disappear into the surroundings, you feel like you own this world.  All you need is a noise cancelling pair of Bose. Al right, i am kidding.

Its a very personal experience i am writing about today. As days pass by, i am getting this sense of alienation. Family, acquaintances, peers – everyone seem so repelling. Its like our individual agenda are conflicting. And like she would say, there is no merging. Have you ever felt so disillusioned that you learn to ignore everything good or bad? Have you questioned yourself whether you have ever done a correct thing? Felt like a looser all the way down?

May be you actually are. May be not. I am still quite hopeful and hence i shall take the latter stand. In this world full of mediocrity its easy to get into conflicts with things which have been recognized by the mass as convention, rules and regulations. Its then you begin to question yourself: are you wrong or have you outgrown the mass? Are you the tall tree who have been enterprising enough to reach out?

Outgrowing has interesting repercussions aside from alienating you. It makes you a misfit of almost villainous proportions. You are made to look like a traitor to your roots. Outgrowing often makes you the object of a chaining project.

The key to maintain growth is to maintain the upward climb. The higher you grow, the farther you live mediocrity behind. Its not unknown that lion does not move in a herd. The lion possesses beastly physical powers. Most human beings don’t possess even a fraction of that physical strength cause we are not all flesh and bones. We are supposed to be thinking beings and its our thoughts which should empower us above the mass.

Now you might be wondering what is the definition of outgrowing that i am talking about. There is no particular definition i have in mind. Its a feeling, more of a realization of not belonging to your immediate community, surroundings and family.

This year has taught me in a very hard way what mediocrity can do. Shun people who make you feel mediocre. You are awesome.

May you outgrow all during the course of next years.

Okay, this post comes at a time when December is already upon us, Fedora 16 has been released for a month now and that also means that Fedora Scientific has seen the light of the day for a month now. I felt this might be a good time to describe the current state of the project and my plans for the next release(s).

Software in Fedora Scientific (Fedora 16)

The current list of software available in Fedora Scientific is available here [1]. Briefly, they are:

Scientific Computing tools and environments: The numerical computing package GNU Octave, front-end wxMaxima, the Python scientific libraries SciPy, NumPy and Spyder (a Python environment for scientific computing) are some of the software included in this category. A development environment for R, the statistical computing environment, is also included, and so are the ROOT tools for analysing large amounts of data.

Generic libraries:    Software in this category includes the GNU C/C++ and FORTRAN compilers, the OpenJDK Java development tools, and the IDEs NetBeans and Eclipse. Also included are autotools, flex, bison, ddd and valgrind.

Parallel  and  distributed programming   tools / libraries: Software tools and libraries included in this category include the popular parallel programming libraries OpenMPI, PVM, and the shared-memory programming library OpenMP. Also included is the Torque resource manager to enable you to set up a batch-processing system.

Editing,  drawing  and  visualisation  tools: So you have simulated your grand experiments, and need to visualise the data, plot graphs, and create publication-quality articles and figures. The tools included to help you in this include LaTex compilers and the Texmaker and Kile editors, plotting and visualisation tools Gnuplot, xfig, MayaVi, Dia and Ggobi , and the vector
graphics tool Inkscape.

Version control, backup tools and document managers: Version control and back-up tools are
included to help you manage your data and documents better: Subversion, Git and Mercurial are available, along with the back-up tool backintime. Also included is a bibliography manager, BibTool.

Besides these four main categories, some of the other miscellaneous utilities include: hevea–the awesome LaTex-to-HTML converter, GNU Screen and IPython.

As you can see that the list of software is quite extensive, thanks to the awesome Fedora developers who have packaged this gamut of software.

Future Plans

The current release marks the beginning of a project very close to my heart. I feel that such a spin shall definitely be useful for the current Linux community members and future enthusiasts who use Linux for their computing needs. In the next release(s), I intend to explore the following directions for the spin:

• A GNOME based spin in addition to the current KDE spin
• Custom wallpapers
• Work with the websites team to update the Spin website to include high quality images of scientific software and more content
• Collect feedback from the community and act on it 

Talk to Us, Contribute

Come, talk to us on the Fedora SciTech SIG mailing list [2]. Thanks to all the members of SciTech SIG for their useful discussions and comments.

Acknowledgements

Fedora Artwork and Fedora Websites team for help in the artwork for the spin,  Bill Nottingham for the initial comments on the idea and Christoph Wickert  for seeing the spin through for release. All the other people who contributed even with a single word of encouragement online and offline, please acknowledge my sincere thanks.

References

[1] https://fedoraproject.org/wiki/Scientific_Packages_List
[2] http://fedoraproject.org/wiki/SIGs/SciTech

Parts of this blog post has been reproduced from my article on Fedora Scientific Spin  published in the December, 2011 issue for Linux For You.

When I wrote this [1] article a while back, the intention was to publicize the software tools that I was personally using at the point of time to help me in my research work- plotting graphs, analysing data, writing papers, running simulations, e.t.c. Those tools soon became indispensable for my research and hence I always installed them first after a fresh install of Linux. I longed for a Linux distro which would already have these tools installed and allow me to have a fully functional Linux workstation from the first boot.

The Scratching begins

I was getting wary of Ubuntu after their last release (April, 2011) and was looking for a new distribution to commit to – I thought I will give Fedora a shot (last time I tried Fedora was during the Fedora Core days) on one of my computers. Then, I started looking around for ways to create custom Fedora spins when I came across the tutorial for Fedora [2]. And that’s pretty much all I needed to get started working on a Linux for users in Science and Academia – Fedora Scientific

Discussions on Mailing lists

The most fruitful technical part of the discussion happened on the Canberra Linux User’s Group. [4] Thanks to all the folks who made suggestions for various packages and more importantly opined that the spin would be useful to the target audience.

Fedora Spins SIG

The official word on whether the proposed spin would be found useful by the Fedora community in general and Linux community overall was decided by the Fedora Spins SIG  [5]. Thanks to their support and approval.

Where next

Fedora Scientific is officially on course for release with the Fedora 16 release in the next few days. The nightly builds are now available from [6]

Talk to Us, Contribute

Come, talk to us on the Fedora SciTech SIG mailing list [7]. Thanks to all the members of SciTech SIG for their useful discussions and comments.  This page explains the spin in more detail.

Current List of Packages

The current list of software made available in Fedora Scientific Spin are at [8].

Acknowledgements

Fedora Artwork and Fedora Websites team for help in the artwork for the spin,  Bill Nottingham for the initial comments on the idea and Christoph Wickert  for seeing the spin through for release. All the other people who contributed even with a single word of encouragement online and offline, please acknowledge my sincere thanks.

Links

[1] linuxgazette.net/173/saha.html
[2] http://fedoraproject.org/wiki/How_to_create_and_use_a_Live_CD
[3] https://fedoraproject.org/wiki/Scientific_Spin
[4] http://lists.samba.org/archive/linux/2011-July/030331.html
[5] http://fedoraproject.org/wiki/SIGs/Spins
[6] http://dl.fedoraproject.org/pub/alt/nightly-composes/
[7] http://fedoraproject.org/wiki/SIGs/SciTech
[8] https://fedoraproject.org/wiki/Scientific_Packages_List

In the next post, which I intend to do soon after the official release, I shall talk about the applications and programs installed in Fedora Scientific.

And last, but by no means the least- Snowy, you make this world a better place for me.

Like I have mentioned elsewhere, I like to introduce complex (and not so complex) concepts in a popular science fashion – for consumption by even the high-school kid. Hence, when I was contacted by Bella from The Conversation, I was excited to write about Evolutionary Algorithms and optimization and what we as a group work on.

The article is now live here. Hope you enjoy the read. Many thanks to the team at The Conversation for the final touches on the article.

Snowy, your support in various ways is always appreciated.

Link: http://theconversation.edu.au/explainer-evolutionary-algorithms-3580