PiCloud + Pyevolve: Evolutionary Algorithms in the Cloud


It was quite simple to get Pyevolve running up on PiCloud.

For an Evolutionary Algorithm, there are a number of ways it can be parallelized. One of the easiest things to do is to run parallel instances of the algorithm with different initial random seeds. (This is a common practice in the research community – since starting with different random seeds allows testing of the robustness of a new algorithm or the correct  implementation of an existing one). The first exercise that I tried with Pyevolve + PiCloud is to run Pyevolve’s Genetic Algorithm implementation with 10 different initial seeds on PiCloud  – all running in parallel. Here’s how, you can go about it:

Installing Pyevolve

  1. Clone the git repository from https://github.com/perone/Pyevolve
  2. Install Pyevolve: $ sudo python setup.py install
  3. Just to check if it has been installed properly, just run one of the examples in the examples/directory

Setting up Picloud

  1. To setup PiCloud, please follow the instructions at http://www.picloud.com/ starting with registering for a free account
  2. Once you have installed the PiCloud client, please verify the installation by typing ‘import cloud’ from your Python prompt and see that it doesn’t throw up any errors

Getting Ready to Launch

For the purpose of this experiment, I shall be using one of the examples that ship with Pyevolve: pyevolve_ex7_rastrigin.py after some modifications. The modified file is here. Specifically, we change the main() function to run_ga() taking two parameters: seed and runid. The seed parameter is used to provide a random initial seed (used when creating the GA engine using ga=GSimpleGA.GSimpleGA(genome,seed)). The CSV adapater is initialized to store the statistics in a CSV file, where the runid file is used for specifying the name of the file.

Okay, now I shall describe the main driver script which I used to run this GA in the cloud. Here it is:


#!/usr/bin/python
# Simple demo to show how to run the Pyevolve
# Evolutionary Algorithms framework on PiCloud
# Pyevolve: http://sourceforge.net/projects/pyevolve/
# PiCloud: https://www.picloud.com/

# Amit Saha
# http://echorand.me

from pyevolve_rastrigin import *
import cloud

cloud.setkey(<API KEY>, 'SECRET KEY')

# List of Random seeds and run-Ids
# assuming 10 runs
seed_list=[100*(i+1) for i in range(10)]
runid_list=[i+1 for i in range(10)]

# calls the method defined in pyevolve_rastrigin.py
# which initiates the GA execution.
# Execute the code on PiCloud
jids = cloud.map(run_ga,seed_list,runid_list)

# check if the jobs are complete, if yes
# pull the stat files
cloud.join(jids)
for i in range(10):
    cloud.files.get('stats_' + str(i+1) + '.csv','stats_' + str(i+1)+'.csv')

One of the first things you need to do is set the API Key and Secret key that you will be using to access PiCloud. The API Key and the Secret key can be seen in the “API Keys” page of your PiCloud account. This is done by the line cloud.setkey(, 'SECRET KEY'). Next, we use the cloud.map() function to call the run_ga function for each pair of the values of the lists seed_list and runids_list. This is a efficient way of running the same function in the cloud, but with a different set of parameters. Once that is launched, you can see the state of your jobs from your PiCloud’s account’s Jobs page.

Next, what we do is we wait for all the jobs to finish using cloud.join(jids) and then pull all the statistic CSV files from PiCloud’s storage to the local file syste using cloud.files.get('stats_' + str(i+1) + '.csv','stats_' + str(i+1)+'.csv') (More information on using the cloud.files module is here). Once the files have been copied, you can see the results from the CSV files – each CSV file representing the result of one run of the GA. We have not yet talked about how the CSV files were created, however.

Creating files on PiCloud from Pyevolve

The source code of CSV adapater, where the CSV files are created is in pyevolve/DBAdapters.py. The open() method of class DBFileCSV is where the file is opened for writing and the close() method of the same class closes the file handle when the writing is finished. However, this method of creating files won’t work for PiCloud – rather the data won’t be retrievable for our client. We have to create the file the PiCloud way – which is to use the cloud.files.put() function. All I did here was add the line cloud.files.put(self.filename) in the close() method. And I reinstalled Pyevolve module and it all worked fine. You may find my modified DBAdapaters.py file here.

Conclusion

I hope to discuss with the Pyevolve folks what they think of my changes to the DBAdapter class and see if they suggest a better way. Like I mentioned in the beginning, this is a very naive way of harnessing the power of parallel computing in Evolutionary Algorithms. I hope to explore more in this direction with PiCloud. If you have any queries, please leave a comment.

Thank you for reading.

About these ads

65 thoughts on “PiCloud + Pyevolve: Evolutionary Algorithms in the Cloud

  1. Hello,

    A better way to do the trick on 1 solve run of pyevolve,
    Will be to modify GPopulation.py part of the multiprocessing usage.
    and change the function pool.map by cloud.map :
    ———————————-
    def evaluate(self, **args):
    “”” Evaluate all individuals in population, calls the evaluate() method of individuals

    :param args: this params are passed to the evaluation function

    “””
    # We have multiprocessing
    if self.multiProcessing[0] and MULTI_PROCESSING:
    logging.debug(“Evaluating the population using the multiprocessing method”)
    proc_pool = Pool()

    # Multiprocessing full_copy parameter
    if self.multiProcessing[1]:
    results = proc_pool.map(multiprocessing_eval_full, self.internalPop)
    for i in xrange(len(self.internalPop)):
    self.internalPop[i] = results[i]
    else:
    results = proc_pool.map(multiprocessing_eval, self.internalPop)
    for individual, score in zip(self.internalPop, results):
    individual.score = score
    else:
    for ind in self.internalPop:
    ind.evaluate(**args)

    self.clearFlags()
    ———————————

    I think its an easy task to modify it. I will do it this night :-)

    Thanks for your idea.

  2. I was wondering if you ever considered changing the layout of your
    blog? Its very well written; I love what youve got to say.
    But maybe you could a little more in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having 1 or two pictures.
    Maybe you could space it out better?

  3. In addition to this, as there is a healthy progression that can manage determination as
    you can physically see your abilities develop. When the Samurai class slowly faded, some Samurai developed the discipline in order to keep the
    values and philosophies alive. When comes to world of mixed martial arts no one can quite say where its headed.

  4. Nice post. I used to be checking constantly this weblog and I am impressed!

    Extremely useful info specifically the ultimate
    section :) I maintain such info a lot. I used to be
    seeking this particular info for a very long time. Thank you and best of luck.

  5. Rigbht now it appears likee BlogEngine is the
    top blogging platform available right now.
    (from whqt I’ve read) Is that what you’re using on your blog?

  6. Play a single player campaign, go online with or against your friends, even versus the world, in competitive
    and cooperative levels. With the Poo – Yoos, a merry band of adorable baby animals, children will have fun while learning the
    basics for his or her development: numbers, letters, space,
    colors and more. The realization that the League
    has had this weapon hanging over us from space this whole time would
    prove to be one of their most controversial manuvers, and a major
    justification for Cadmus’s distrust of the League’s actions.

  7. One more important concept is what would help us be fit when it comes to martial arts.
    If you are not comfortable training with them, you may want to look elsewhere.
    The best advice is to go yourself and watch the classes yourself.

  8. The Internet these days is thriving with all kinds of file types including
    archived files. The software performs fast and flawless
    recovery of Word files. Next, prepare longer a photograph
    Pattern to improve the texture.

  9. “Since this association began, an impressive 98 percent of our members have remained part of the MMACA, which can lead to long-term income potential for our affiliates and partners,” said John Spencer Ellis, founder
    of the Mixed Martial Arts Conditioning Association (MMACA), which
    offers mixed martial arts certification for coaches and trainers.
    They should also observe a few classes to become familiar with each instructor’s
    training style. This direction is true for the roundhouse in all variations of the kick at least to some degree.

  10. I do accept as true with all of the concepts you have presented in your post.

    They’re really convincing and will definitely work. Nonetheless,
    the posts are too quick for starters. May you please extend them a little from subsequent time?

    Thanks for the post.

  11. In 1983 Dr JD Robinson introduced the \”Robinson formula\”.

    These help the young ones to learn how to do basic things such
    as walking, talking, and learn how to use the bathroom by potty training them.
    Perhaps the most important and mildly game-breaking skill to have in your
    Alpha Protocol career builds is the Stealth based “Awareness”.

  12. Headsets are reasonably affordable, and offer much greater value
    for money. You do not have to travel to Vegas to an actual physical
    casino to play the casino games there. The banker may employ behavioral and statistical analogy before offering brides to
    send the player home.

  13. Hello! Quick question that’s entirely off topic.
    Do you know how to make your site mobile friendly? My website looks
    weird when viewing from my apple iphone. I’m trying to find a template or
    plugin that might be able to fix this issue.
    If you have any recommendations, please share.
    Thank you!

  14. It is not of course expected of a trainer to be acting
    like a clown but there is always a way of tickling the interest
    of a young mind. As with any other profession, just because a teacher
    is highly qualified and even has a wall full of
    awards and trophies doesn’t qualify them to be a good teacher.
    The finest college in the planet won’t do you any very good if it is also far
    away for you to make courses often.

  15. Heya i’m for the primary time here. I came across this
    board and I in finding It really helpful & it helped me out a lot.

    I am hoping to provide something again and aid others such as you helped me.

  16. The movie does a fantastic job of capturing a rising sense of unease proper out of the
    gate, from the family camping within the wilderness and being woken up
    by unexplained lights darting around the evening sky,
    to coming across flocks of dead birds on the road, to having repeated
    automotive issues, proper up to the second half of the movie which is pretty
    much a non-stop chase sequence because the household runs via the
    woods in the middle of nowhere, making an attempt to get away from these seemingly all-powerful
    aliens that are after them; Should you’re the form of person that is prone to panic attacks, chances are you’ll wish to schedule your self
    some breather breaks if you sit down to look at
    this film, because you will need them and once shit hits the fan, the movie doesn’t let up or
    provide you with a chance to catch your breath in any respect.

  17. Next up was another beach on the south side
    of the island, sa Caleta, just west of the airport.
    It’s a fact though that most of us still aren’t drinking enough water to help our bodies and brains achieve maximum performance, and most people
    fall far short of the recommended 8 glasses a day.
    5 age bracket flora, your Guangzhou Municipal National young
    optical across, is.

  18. Most likely the relentless tapping itself taps into how
    people experience their meaningless lives; lacking purpose in the role of one
    to ages of automation and redundancy of purpose; the relentlessly advancing pipes of death remind us that our life is fleeting, and how we live at the moment are
    spent navigating redundant technology: online is constructed of pipes.
    The article states “I see Obama’s going to the United States,” said Rush Limbaugh on Thursday, the president’s first full day in Washington following
    a springtime diplomatic tour of Latin America. Although Revenants fluctuate in stage,
    if you are beneath stage 100, it’s best to avoid the Revenant as it could possibly
    easily kill you and gives little or no expertise when you do manage to defeat it.

  19. If you want on while using Wiis backwards compatibility then this Gamecube controller is crucial, they are often bought on e – Bay
    for peanuts. Besides improving memory and aptitude for learning, math
    games could also boost your child’s confidence within the subject
    and prepare them for further challenging learning experiences later in life, if not
    they might still find it hard later on. Onlineschooladmissions is often a portal that
    serves parents and helps makes school admissions a delightful experience.
    Some games contain the memory, always has those classic games can hardly
    be forgotten. Today, you are able to build a game on just about any topic you need.
    Cube Crash, Bomboozie and Lynica are all great samples
    of some from the fun puzzle games you can find and
    obtain addicted without difficulty using their interactive play.

    People can now join communities of numerous genres through forums,
    social websites networks, blogs and websites. To play, the
    gamers should manage to pick gifts from the conveyor belt and place them
    correctly in the horizontal row position into the train cars.
    If you are a basketball lover, it is possible to play Slam Dunk or 3 Point Hoops.
    Were talking about a game-cum-crash length of interior designing.

    However, you will find always adjusters available if
    you do need any help. These forms of pursuits are obtainable by using an website for
    numerous alternatives. As players progress they can earn various power-ups and tools.

    The more accurate you snipe the greater incentive points you receive.
    The Ville is the amount lone limitless sport in the moment
    according to game rating spot. The positive part of bingo is that often the concept demonstrates you ones mathematical activities for youngsters don’t should contain sitting on the table with a pencil available.

    We moved to Florida 3 years ago and inherited a children’s pool when purchasing our home.

    Almost these will also ask you to register on his or her site where the instructions
    might be very confusing. This game supplies the education plus the excitement simultaneously meaning that
    your kids eventually will unlock their adoration for
    learning. Funny games on these game portals would be the products of media-driven applications that will make their vibrant
    look and feel very attractive to visitors. World of Warcraft may be the biggest miltidimensional games
    within the world and take hour upon hours to level.
    A host of unnecessary gameplay tweaks, some that completely erased months and a lot of progress for players, were the final nail
    in the coffin with this squandered opportunity. Playing games online won’t cost you money because many websites
    offer this particular service at no cost. In addition to this, you can also make better money by running down more
    zombies inside a single go or going through the hurdles at high speeds.
    It is often a crucial part of review that many of the company’s concept is generally integrated on lifestyle model.

  20. video game developer Maxis, the creator of the blockbuster game Sim – City, goes public.
    You can switch to them to use a bow and arrow, defeat large enemies, use a large spear, etc.
    In China, the game places flesh on skeletal characters and depicts character corpses as small, neat, graves to “promote a healthy and harmonious online game environment.

  21. The first country in which it was found that the new addiction, was China, and there also appeared in its English name: mobile phone dependence syndrome.
    Then we have in my opinion the best two modes in the game.
    With the advent of small downloadable games, companies have slowly rolled out downloadable titles for all of
    the current generation of video games.

  22. 99Disney Junior Magazine #15 (Disney Publishing Worldwide), $4.
    Whether the title is coming from a third-party,
    second-party, or first-party, Sony has a title that’ll interest
    everyone and, more or less, guarantees that you’ll be purchasing at least
    one title this holiday season. Aiding him in his campaign are the
    Titans -a race of mountainous creatures who have a bit
    of a bone to pick with Zeus and his divine cohorts.

  23. By training your favorite video game for cost-free at home, it’s
    possible to figure out how to perform perfectly. The Hacker group ‘Anonymous’ has threatened
    that they will soon kill Facebook, the world’s no.

    To stop your Steam client from automatically patching your game,
    use the following steps.

  24. The Sphinx at Giza (Egypt) is another construction out of solid
    rock of something that’s biologically impossible.
    Remember when your mom brushed your hair when you were
    a child. Chef’s Award ‘ These are the currency received for doing the cataclysm cooking dailies in either Stormwind, Ironforge, Darnassus, Orgrimmar, Undercity or Thunderbluff.

  25. Great goods from you, man. I’ve understand your stuff previous to and you are just too
    excellent. I really like what you’ve acquired here, certainly like what you are
    stating and the way in which you say it. You make it enjoyable and you still take care of to keep it sensible.
    I cant wait to read much more from you. This is really a terrific web site.

  26. Thank you for another fantastic article. The place else could anybody get that type of information in such a perfect approach of writing?
    I have a presentation next week, and I’m on the look for such info.

  27. Some levels are even known for their brutal difficulty,
    which is why so many people are stuck on that one level for
    weeks or maybe even months. Now try the same exercise with each of
    the following:. The particular riddle involving Candies Crush’s growth is within their story acclimatization skeleton,
    which often repays resolve giving people another lifetime every 30 mins, approximately five total.

  28. Remotely monitoring premises using internet protocol (IP) camera equipment
    is a relatively new technology arrival even more so with
    wireless equipment capable of pan-tilt-zoom (PTZ) with wireless connectivity.
    Step6: Reboot computer and login to the locked PC without password.
    Articles of information need to be built around keywords in order to
    bring in the free traffic from the search engines.

  29. The only viruses you’re totally unprotected against are the unknown ones,
    and the only way for your network hosts and servers to
    know about new viruses is to keep them updated. Most companies
    simply jump in options trading, with or without experience
    in marketing. Always take your time going through the site to see what assets the broker has available.
    Having the ability to monitor and plan your exit will help you to optimize your profits and cut your losses.
    Selling away (NASD Rule 3040) l Broker-Dealers l Market Makers.
    If you are thinking on joining binary options trading
    to gain revenues, the first thing that you must do is study the fundamentals
    of options trading.

  30. What could be more frightening than what Ken has in his basement.
    Blu-ray Deal of the Week: RVBX: Ten Years of Red
    vs. The latest e – Book guide sells at the cost of many of those extras on the new game,
    yet provides helpful tips and tricks for beating tougher levels, and extra resource links
    for more help.

  31. Now a days affiliate marketing can be a vary lucrative and money making online business only
    if you have the patience to go with it. It is best to personally be known in the business
    rather than just let the products that you endorse
    be the ones popular. Develop ways to be able to capture their eyeballs and place
    your own offering before all of them. Make sure your affiliate programs are
    related to your website content. For a more in-depthlook at some of the
    programs out there, have a look at this blog which also
    haslinks to many free resources. Sign up with those sites that are offering these
    free services and start posting ads about your
    product by driving targeted traffic to your website.

  32. After checking out a few of the blog articles on your website, I truly like your way of blogging.
    I book marked it to my bookmark site list and will be
    checking back soon. Please check out my web site as well and let me know how you feel.

  33. It’s actually a nice and helpful piece of info. I’m satisfied that you simply shared this useful information with us.
    Please keep us up to date like this. Thanks for sharing.

  34. Chicken thighs are a cheaper part of the bird and I find
    them on sale quite often. I got this gift from him on our wedding anniversary which was
    round the corner of the week. It has been designed so compactly and carrying very small working units
    so that it can be kept anywhere taking a less kitchen space.

  35. Just wish to say your article is as surprising. The clearness in your post is just spectacular and i
    could assume you’re an expert on this subject. Fine with your permission allow
    me to grab your feed to keep up to date with forthcoming post.
    Thanks a million and please continue the gratifying work.

  36. I do believe all the ideas you have offered to your
    post. They’re really convincing and will certainly work.
    Nonetheless, the posts are too brief for beginners.

    Could you please prolong them a little from subsequent time?
    Thanks for the post.

  37. Marvel’s Thor #1, featuring a female taking up the hammer and helmet of the iconic
    character, leads the slate of new comic book releases for Wednesday, October 1, 2014.
    95Batman Essentials Batman The Black Mirror Special Edition #1, $1.
    This is the second in a series of seven articles about building emotional resilience.

  38. Using the tool, you will be able to find out
    exactly what information on your site is being indexed
    by the search engine crawlers. Step6: Reboot computer
    and login to the locked PC without password.
    Anyone who witnessed his backhanded acceptance of a Lifetime Achievement Award at the Golden Globes knows that
    Robert De Niro is a character.

  39. There is absolutely no way of knowing which websites deliver unless you depend on recommendations from people that have used
    them before. You can watch your i – Pad Video Lessons at your
    own convenience. It expects that e-books should be cheaper than their printed counterparts.

  40. Search and research all the keywords that have an interest in your site theme.
    This helps water cool more quickly and increases the rate of evaporation. Articles of
    information need to be built around keywords in order to bring
    in the free traffic from the search engines.

  41. Great web site you’ve got here.. It’s hard to find high
    quality writing like yours nowadays. I really appreciate individuals like you!
    Take care!!

  42. Once you have put in place your online messaging software, alter
    the code to some thing unique. He was very ruthless in pressing his cause, and had the blood of the
    innocent on his hands. Using a bluetooth for a cellular phone leads to battery to drain much more easily.

  43. They even have a community where Super – Docker users have uploaded their own creations
    that you can download. Although boys love a variety of different games and their focuses, it is
    important to ensure that the game that you are choosing
    for them is suitable and will be easily enjoyed.
    If you aren’t interested in spelling what about
    a singing competition.

  44. After checking out a few of the articles on your web page, I truly appreciate your way of
    blogging. I saved it to my bookmark webpage list
    and will be checking back soon. Please visit my web site too
    and let me know how you feel.

  45. Thanks a bunch for sharing this with all people you
    actually realize what you’re speaking about!
    Bookmarked. Please additionally discuss with my website
    =). We may have a hyperlink alternate arrangement
    among us

  46. Although Facebook is a big, notable company it
    is still a young pup in the internet world.
    A person’s email address is also one of the most commonly required personal information in government forms, applications
    and surveys. The game begins to develop throughout popularity through
    This summer regarding 1986 it really is hard-wired to the Apple 2 and also Commodore 64.

  47. Just as “Afterlife With Archie” took a dark and
    creepy turn for the characters of the Archie series, this did the same thing
    for Sabrina. Being a first issue, the character origins start
    this book off, and the nice thing is that it’s set in the 1950’s.
    Little is known about Vlad except that he was cast into
    the garbage as a baby and taken in by a kind Butcher.

  48. ” Thanks to Facebook Join it is possible for you to to access our help channels, work together on the Q&A bit, like articles and assist our engineers together with your feedback. On Zynga Poker On line casino Gold & Chips Hack oahu is the same. Ive lost at furniture where players know the other Zynga Poker Casino Platinum & Chips Hack go all in deliberately to get your motherboards.

  49. You can actually trade as little as $1 for every single deal.
    It is people who complicate it more than it really is. That is where trading becomes a boring process but eventually it
    brings very good profits. In an upcoming article, I will
    talk more about Banc de Binary and its investments. It is very impotant be ready to commerce as soon for
    t reason that knowledge is printed. If the investor is fearful a stock could fall within a
    particular time, he can set up a set option to shield downward threat.

  50. The system is very simple and is ased on the concept off virtual partitioning
    a server into several segments and designating one segment per website.
    Like dynamic IP or TCP addresses, these setups aree made to shield online users activities from
    stazrt to finish. No longer did a company need to lease dedicated point-to-point circuits, all they needed wdre
    these devices, Internet access, and they
    were set.

  51. A car dealer licene iis usually needed but the documents needed
    to obtain it will differ from state to state
    (country to country). Today, there arre other private organizations emerging too sponsor this activity
    thus attracting more participants. However, if it is jhst a scrap car, you
    may get little foor it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s