tag:blogger.com,1999:blog-30070548649877599102024-03-12T20:07:26.349-05:00Derek WeitzelSolutions to computer problems.Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-3007054864987759910.post-13810957904525299792013-08-30T17:55:00.004-05:002013-08-30T17:55:55.140-05:00You asked, we delivered!<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCoDuYrC2hHoeSxNaHsWyeQ-cx8d5hWgR4jc89ZC2zkFIsHg2VU-08cqfmXDuwCtxz4VOdxKhsqFWlqO_pv4fJwY_0D3VecxGHO_YZFRdmT0G_4j8s6cJTg1S_y1kRZj_uAZaaPt_2cVjS/s1600/BoscoR-pic1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCoDuYrC2hHoeSxNaHsWyeQ-cx8d5hWgR4jc89ZC2zkFIsHg2VU-08cqfmXDuwCtxz4VOdxKhsqFWlqO_pv4fJwY_0D3VecxGHO_YZFRdmT0G_4j8s6cJTg1S_y1kRZj_uAZaaPt_2cVjS/s640/BoscoR-pic1.png" width="640" /></a></div>
<br />
<br />
A few users of <a href="http://bosco.opensciencegrid.org/boscor">BoscoR</a>, and specifically <a href="http://osg-bosco.github.io/GridR/">GridR</a>, have requested several major features:<br />
<ol>
<li>Allow custom packages to be installed on the remote worker node (<a href="https://github.com/osg-bosco/GridR/issues/3">#3</a>).</li>
<li>Specify custom R binaries for the worker nodes (<a href="https://github.com/osg-bosco/GridR/issues/4">#4</a>). Along with custom binaries, we also added logic so that if you upload new binaries to a URL, they will automatically be detected and installed (<a href="https://github.com/osg-bosco/GridR/issues/7">#7</a>).</li>
<li>Asynchronous updating of batched processing. This will enable the results to show up in the result list before the entire processing has been completed (<a href="https://github.com/osg-bosco/GridR/issues/6">#6</a>)</li>
</ol>
Along with the above mentioned issues, we have also updated the default download location to be hosted at Indiana rather than Dropbox (hence the earlier <a href="http://derekweitzel.blogspot.com/2013/08/how-you-know-you-are-doing-something.html">blog post</a>).<br />
<br />
I hope to write a few posts in the next weeks with a more in-depth analysis of these significant user facing improvements, but in the mean time you can read the <a href="https://github.com/osg-bosco/GridR/releases/tag/v0.9.8">release notes</a> which include details on how the new features work and how you can use them.<br />
<br />
Today we have released the new version, <a href="https://github.com/osg-bosco/GridR/releases/tag/v0.9.8">v0.9.8</a>, which includes all of these changes.Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-23633626756658976242013-08-30T13:55:00.003-05:002013-08-30T13:55:57.599-05:00How you know you are doing something right?I just received this from <a href="https://www.dropbox.com/">Dropbox</a>:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNwuYeqiXjK2O8oALTxvHZQFijEXz6GqRMgGsuLgBWvuoU6yb7y4SWr2lAg1teIgv5wf4gUXl74_VRSnwRmMlvjTh-V7nH_kEx1LIvIzwxC-bJejztLELh3QT3O3ksxRinoml5RqOcwUVn/s1600/Screen+Shot+2013-08-30+at+1.52.30+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="626" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNwuYeqiXjK2O8oALTxvHZQFijEXz6GqRMgGsuLgBWvuoU6yb7y4SWr2lAg1teIgv5wf4gUXl74_VRSnwRmMlvjTh-V7nH_kEx1LIvIzwxC-bJejztLELh3QT3O3ksxRinoml5RqOcwUVn/s640/Screen+Shot+2013-08-30+at+1.52.30+PM.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The notification!</td></tr>
</tbody></table>
An R user in Wisconsin was using GridR so much... that he used up my bandwidth limit on Dropbox while downloading the R binaries. I'm going to put this down as "a good problem to have".<br />
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-19916352051863683062013-08-28T19:00:00.004-05:002013-08-28T19:00:52.405-05:00CIC 13 MeetingThis week, the <a href="http://www.campusgrids.org/">Campus Infrastructures Community</a> debuted the <a href="https://osgconnect.net/">OSG Connect</a> service. Overall it was a very successful workshop. I met 2 users that where very excited about <a href="http://bosco.opensciencegrid.org/boscor">BoscoR</a>, and we had great discussions about using the OSG.<br />
<br />
Here are a few pictures from my trip to Duke for your viewing pleasure:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2wPEiOzExKldTZ0Oaz_LtpL0fpnqGMQxoXS_u8VBBuCH3PNZn89CudNvkTm_dj5i0P5DRP3reGsyB-zWUk9Uc820Ta85Z1T5KrY8rHeyWpBk4MZggzrBObkLlcDcgu8zFwHQb1xYgG5aQ/s1600/2013-08-27+13.40.38+HDR.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2wPEiOzExKldTZ0Oaz_LtpL0fpnqGMQxoXS_u8VBBuCH3PNZn89CudNvkTm_dj5i0P5DRP3reGsyB-zWUk9Uc820Ta85Z1T5KrY8rHeyWpBk4MZggzrBObkLlcDcgu8zFwHQb1xYgG5aQ/s640/2013-08-27+13.40.38+HDR.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mats Rynge teaching the basics of High Throughput Computing</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgH3F09DQ_a89r5hKeGDXAYZ9NB0tIunJOzLCob_BBu5mecMjExCIlUiEbGd5I5ar0E4kX2mZRjN1z3DWFDsKsagP3klS0diTV5W9Idm-Hab3JOiEBUbsIN7aub795fXDeImuPhACXHHnpQ/s1600/2013-08-27+19.15.06+HDR.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgH3F09DQ_a89r5hKeGDXAYZ9NB0tIunJOzLCob_BBu5mecMjExCIlUiEbGd5I5ar0E4kX2mZRjN1z3DWFDsKsagP3klS0diTV5W9Idm-Hab3JOiEBUbsIN7aub795fXDeImuPhACXHHnpQ/s640/2013-08-27+19.15.06+HDR.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Wandering around Duke University</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmUvKhLuojxawWRzLw_qnf0guSSckIc-6bmfSxJEto_8_K52-GpTecWvQcVgbv36UFmy2mnZaDdLldsV8tiKjblS5AL8CYPDFznOj1Qopw9EmQiyYqqK9DyEcwzXI19R0AZXyq_cisooBi/s1600/2013-08-27+19.25.48.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmUvKhLuojxawWRzLw_qnf0guSSckIc-6bmfSxJEto_8_K52-GpTecWvQcVgbv36UFmy2mnZaDdLldsV8tiKjblS5AL8CYPDFznOj1Qopw9EmQiyYqqK9DyEcwzXI19R0AZXyq_cisooBi/s640/2013-08-27+19.25.48.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The "Football" stadium at Duke. I won't make any comparisons to Nebraska's...</td></tr>
</tbody></table>
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-91511057121641469292013-07-11T15:34:00.003-05:002013-07-11T15:34:16.997-05:00Creating a native Mac Installer for BoscoIt has been our goal for some time to create a native installer for our supported platforms. This week, we created a prototype for the first of those platforms, the Mac installer.<br />
<br />
<h3>
The PKG file</h3>
<div>
The first step to creating the installer is the PKG file. I installed Bosco the usual way, using the multi-platform installer. Then, I looked through the Bosco files (using a find command) to locate all of the instances of hard coded paths. They where located in the bosco.sh, bosco_setenv, and the condor_config files. I modified those to use the environment variable HOME.</div>
<div>
<br /></div>
<div>
Then I used <a href="https://developer.apple.com/downloads/index.action">PackageMaker</a> from the auxiliary tools for Mac package available on the Mac developer page (free registration required). In PackageMaker, I opened the Bosco directory in the home directory.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguexavEvBTJWv-W79ktHGaZlxR9jIxEGIvimOytO2fFh2ZQ3vEeNNkYQxXTYPNkyJ9onGUmLehvJzBK7H9wYdRoX3C9sVZ3xzcOX8aIgrOMpml67dP3JLAHOL96dyfnUok7nexFHg0wTYx/s1600/Screen+Shot+2013-07-11+at+3.26.29+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="529" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguexavEvBTJWv-W79ktHGaZlxR9jIxEGIvimOytO2fFh2ZQ3vEeNNkYQxXTYPNkyJ9onGUmLehvJzBK7H9wYdRoX3C9sVZ3xzcOX8aIgrOMpml67dP3JLAHOL96dyfnUok7nexFHg0wTYx/s640/Screen+Shot+2013-07-11+at+3.26.29+PM.png" width="640" /></a></div>
<div>
I unchecked the 'Require admin authentication' because bosco will be installed in the user's home directory. After these simple steps, I built the installer and tested, everything worked!</div>
<div>
<br /></div>
<h3>
DMG File</h3>
<div>
A DMG is a disk image that is used for software distribution. The first step is to create the Bosco.dmg disk image using Mac's built-in Disk Utility.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkrlsjt9OkePOOuEhlEflW9GW11TeuypbQ_S5Uo3BneeOemo9QRmoarNtVxPr9cezKtPfBjMXZG3Na_uUDy7Lo8hK9tY8IHbsY2lnvI8S_a5gIxsQBiGayBhhzcGJNj7PGkp_b77JTN7eE/s1600/Screen+Shot+2013-07-11+at+3.29.30+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="564" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkrlsjt9OkePOOuEhlEflW9GW11TeuypbQ_S5Uo3BneeOemo9QRmoarNtVxPr9cezKtPfBjMXZG3Na_uUDy7Lo8hK9tY8IHbsY2lnvI8S_a5gIxsQBiGayBhhzcGJNj7PGkp_b77JTN7eE/s640/Screen+Shot+2013-07-11+at+3.29.30+PM.png" width="640" /></a></div>
<div>
After mounting the DMG, I copied the Bosco installer into the image. Then, I ran the command:</div>
<div>
<pre class="code-java" style="background-color: whitesmoke; color: #333333; font-size: 12px; line-height: 16px; max-height: 30em; overflow: auto; padding: 0px; white-space: pre-wrap; word-wrap: normal;">bless --folder /Volumes/Bosco --openfolder /Volumes/Bosco</pre>
</div>
<div>
<br /></div>
<div>
This will cause the DMG to automatically open a window when mounting it.</div>
<div>
<br /></div>
<h3>
Where to get this</h3>
<div>
The Bosco DMG makes it very easy to install Bosco on a Mac. The DMG is available on Dropbox, and will be put in production when a few more minor fixes are put in place.</div>
<div>
<br /></div>
<div>
<a href="https://www.dropbox.com/s/8ljehuufos3x7hf/bosco.dmg">Bosco.dmg</a></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-43552811827437112842013-07-05T09:00:00.000-05:002013-07-05T18:42:05.891-05:00The next step for Bosco: BoscoRWith the 1.1 release of Bosco in January, the Bosco team finally had a usable product for researchers. It didn't have the best usability, but it was a solid tool to build on. When planning the next release, 1.2, we determined that we would not include many new features and instead focus on usability. With that focus, we interviewed researchers and looked around at our colleagues to see how most researchers interacted with their data on a daily basis, which was not with Condor or Bosco. Researchers used applications such as Matlab, Galaxy, or R. After investigating the options, we determined that R would be a great first application to integrate with Bosco. It is open source, has a strong community, and most importantly, it is heavily used by the researchers around us.<br />
<h3>
The Initial Steps</h3>
<div>
Our first step was to look at how people are submitting R to HTC resources now. The CHTC does a great job of <a href="http://chtc.cs.wisc.edu/MATLABandR.shtml">submitting</a> entire R scripts for Wisconsin researchers. There is also <a href="http://diagrid.org/resources/58">SubmitR</a> from Hubzero which integrates a GUI with submission of R scripts. Both of these methods had good and bad attributes. </div>
<div>
<br /></div>
<div>
The CHTC method was very easy to submit complex R scripts with many package dependencies to a very large number of nodes. But the researcher had to submit only on UW machines with the proper scripts installed. Also, the researcher still had to learn at least a minimal amount of HTCondor commands to successfully run their jobs. The researcher had to leave their environment and learn another, HTCondor.</div>
<div>
<br /></div>
<div>
SubmitR, on the other hand, was very easy to use, with a graphical interface to upload and start jobs. SubmitR is open source, but deeply, deeply integrated with HubZero. Therefore, any submissions to SubmitR would need to be centralized to a small number of locations. Also, SubmitR had the same issue, the researcher had to leave their environment to learn another, in this case, HubZero and SubmitR.</div>
<div>
<br /></div>
<div>
We found that the with Bosco installed on the researcher's laptop, we could leverage that locality to create a better interface. Our next step was to look a possible packages that could submit R jobs to clusters. They are listed on the HPC <a href="http://cran.r-project.org/web/views/HighPerformanceComputing.html">page</a> for R packages. The only package that fit our requirements, open source, released, and in <a href="http://cran.us.r-project.org/">CRAN</a> was <a href="http://cran.r-project.org/web/packages/GridR/">GridR</a>.</div>
<h3>
Integrating Bosco with GridR</h3>
<div>
When first looking at the source of GridR, it can be very intimidating. It is using asynchronous assignments to variables (which I would later learn is frowned upon in the R community). It submits jobs by forking a new processes to do the submission and monitor the jobs. Is uses R on the remote side to load the functions and variables to run the processing.</div>
<div>
<br /></div>
<div>
Even though GridR is complicated, it has one great feature, everything is done from inside the R environment. The researcher simply loads a package and uses a familiar function, apply. With that function, the researcher can send a R function to a remote cluster to be processed, and the result will be written asynchronously back to the environment.</div>
<div>
<br /></div>
<div>
When I began looking at modifying GridR, my first step was to find the source repository. Unfortunately, the package hasn't received an update since 2009, and the source repo was no where to be found. Luckily though, another researcher from Harvard had just modified GridR to submit to R jobs to a locally installed HTCondor cluster. By studying his changes, I was able to <a href="https://github.com/osg-bosco/GridR">fork</a> his work and add Bosco submission to GridR.</div>
<div>
<br /></div>
<div>
The modifications to GridR where minimal. I needed to add a new submission type to package, bosco, and modify the local submit scripts with the Bosco submit settings. But GridR requires R to be installed on the remote cluster, so how is Bosco going to get R to the remote worker node before the job executes?</div>
<div>
<br /></div>
<div>
Several different ideas where floated to install R on the remote worker node. We settled on a solution that used a bootstrap method. When the remote job started, instead of directly executing R, it would run a custom bootstrap script written for Bosco submissions. This bootstrap script would determine if R is already installed, and if not, it will download and install R either in the user's home directory or temporarily on the worker node for the duration of the job. This bootstraping enables GridR submission to successfully run anywhere with network access, whether it's on a campus cluster, or even national infrastructure, such as the OSG or XSEDE.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://bosco.opensciencegrid.org/boscor/"><img border="0" height="79" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwJQA9Svd5HT-5ZHPydM3mFDCl4cf0qdkEsTbu7PUZYT3RZywbfCGi7WjieMg88NkuAK80z5ZxTiB6yugZ9Nqlr2vEmfGS00YsibhlfFI-HmGBJ3Tvi8ioObUBbdIvUpf7E2QnEtvZRbZ-/s320/BoscoR-pic.png" width="320" /></a></div>
<div>
<h3>
Current Status</h3>
</div>
<div>
The first round of alpha testers have reported great success at running BoscoR. From installation to submission, all alpha testers were successful at running R jobs on remote clusters without any help from the Bosco developers. We also got much needed feedback from the R users on how the interface worked for them.</div>
<div>
<br /></div>
<div>
The most common feedback was the use of the apply function for running multiple jobs. The testers are used to the apply function built into the R language, which runs a function once for every element in a list, then returns a list of results. GridR can do this, but it is not the default action of GridR's apply, therefore it provided an inconsistent interface with the rest of the R language. The Bosco team is working hard on this, and it will be fixed in the next BoscoR beta release out next week.</div>
<div>
<br /></div>
<div>
The Bosco team has also submitted the updates to GridR for review by CRAN. Since GridR has not received an update since 2009, many policies have changed as to how packages can interact with the user's environment. As mentioned earlier, asynchronously modifying the user's environment, such as to set a variable to the result of a remotely execute function, is against CRAN policy. I am confident that this can be worked out, but it will take effort to include the updated GridR in CRAN, and it may take a while.</div>
<h3>
Closing Thoughts</h3>
<div>
Integrating with scientific applications and frameworks is the next step in Bosco to increase usability. Bosco cannot remain just a tool to be used, it must have interfaces built on top of it to connect with researchers, to reach out to them. The researchers do not want to leave their environment, and BoscoR provides them a way to stay in their comfort zone, all while completing their research at new levels.</div>
<h3>
Getting BoscoR</h3>
<div>
You can get BoscoR from the Bosco <a href="http://bosco.opensciencegrid.org/boscor/">website</a>.</div>
<div>
<br /></div>
<div>
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img alt="Bosco Download" src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png" style="border-width: 0px;" />
</a>
</center>
</div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-58282213627815265062013-06-05T13:47:00.001-05:002013-06-05T13:47:51.159-05:00HCC OSG WorkshopOver the last 2 days, HCC has been running a workshop on:<br />
<br />
<ol>
<li>Building an HTC Cluster</li>
<li>Running on a HTC Cluster</li>
</ol>
<h3>
Building an HTC Cluster</h3>
<div>
<a href="https://hcc-docs.unl.edu/pages/viewpage.action?pageId=2098110">Documentation</a></div>
<br />
This part of the workshop was very successful. We where able to help small universities across Nebraska build their very own HTC cluster. We went through the installation of Cobbler, Puppet, and Condor. <br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP6tZG5YPPpAZdg9Z0Mr88nA5VyZL7ID7oX0feQuM8kM8XcccO_nR_9oHDqSBDE-gugI_ZJMEfnLlSaPBR4rD67BC1PwcEkUbY7n4G2WXOQOdhRnvXZHLVNUXa149Nj1JLknhDRG48OHQF/s1600/photo.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP6tZG5YPPpAZdg9Z0Mr88nA5VyZL7ID7oX0feQuM8kM8XcccO_nR_9oHDqSBDE-gugI_ZJMEfnLlSaPBR4rD67BC1PwcEkUbY7n4G2WXOQOdhRnvXZHLVNUXa149Nj1JLknhDRG48OHQF/s640/photo.JPG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Building an HTC Cluster</td></tr>
</tbody></table>
<br />
<h3>
Running on a HTC Cluster</h3>
<div>
<a href="https://hcc-docs.unl.edu/display/HCCDOC/June+2013+OSG+Workshop">Documentation</a> - Largely borrowed from the <a href="https://opensciencegrid.org/bin/view/Education/UserSchool12Materials">2012 OSG Summer School</a></div>
<div>
<br /></div>
<div>
We had many more people for the session on running on an HTC cluster. This was another very successful day. The users learned about HTC: how to write a Condor submit file, how to manage workflows, and how to transfer data. We wanted to focus more on the application and general concepts of HTC (the 'theory') rather than the actual technologies. This is why we choose to talk about portability of an application (R), and data transfer using HTTP.</div>
<div>
<br /></div>
<div>
As always, the conversations between the exercises are the most productive. The user's picking the brains of the experts tends to lead to future collaboration.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuFDIijX_s1YrdzpC22aya84M7eCbrDSD-fNooCY7sPISbkwEG67K1XtpMva165GC2Lx7pbhkuBBxyhMxJHRSnzV6lEcKbPOHCesuRl-sB5L2O0BJHBQVBICc4HA5daqloPJ4L-HfN2U5r/s1600/photo.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuFDIijX_s1YrdzpC22aya84M7eCbrDSD-fNooCY7sPISbkwEG67K1XtpMva165GC2Lx7pbhkuBBxyhMxJHRSnzV6lEcKbPOHCesuRl-sB5L2O0BJHBQVBICc4HA5daqloPJ4L-HfN2U5r/s640/photo.JPG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">HCC OSG Workshop</td></tr>
</tbody></table>
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-28723483402510072372013-05-31T09:53:00.002-05:002013-05-31T09:53:26.803-05:00Bosco at GPN Conference<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.dce.k-state.edu/conf/great-plains-network/"><img border="0" height="144" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk38cdh6QkwoeJ8UqhbbRzsa36KiSgE6DtwHSuQTyer57MPM4pdYjh84ZmcFt7M6cloj_sJISNiqqPokR3ueuiuab3iA_wgh_uUmedrgEgR5NM8MtPJyvRsvmXPfUnu68PJkO03qQGZjVU/s640/5882-Great_Plains_Banner.jpg" width="640" /></a></div>
<br />
This week, I presented the Bosco poster at the Great Plains Network annual meeting. It received a very good reception. Many people where interested in what Bosco could do for them. Most of the audience was HPC center admins or directors, therefore they where looking at how they can use Bosco to help their users utilize their campus clusters.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhphPk116Sj0laIJKdisEphUv6wG7aqaneCuoY0T8qkxMN4y59NOFv1gkBfIij8qztM4HdBFgcPDX2pHlUoaKrJDbFQ5AKeMkZOOW1gqND6GfgjpryFF-yH1JaXdWib3dpcmgjSzUUL4CYf/s1600/derek-bosco-gpn.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhphPk116Sj0laIJKdisEphUv6wG7aqaneCuoY0T8qkxMN4y59NOFv1gkBfIij8qztM4HdBFgcPDX2pHlUoaKrJDbFQ5AKeMkZOOW1gqND6GfgjpryFF-yH1JaXdWib3dpcmgjSzUUL4CYf/s400/derek-bosco-gpn.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">At the poser presentation</td></tr>
</tbody></table>
At the conference we heard a lot about networks, as GPN is mostly a network collaboration. But we also heard about the <a href="http://condo-of-condos.org/">Condo of Condos</a> proposal which the OSG is well represented (by <a href="https://en.wikipedia.org/wiki/Miron_Livny">Miron</a>). There is a very good webcast of the description of Condo of Condos on the <a href="http://events.internet2.edu/2013/annual-meeting/program.cfm?go=session&id=10002997&event=1262">I2 Website</a>. It feels like it's very early in the planning phase, but I am curious how the OSG will integrate with the Condos.<br />
<br />
The weather has been terrible, but the meeting has been great.<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com1tag:blogger.com,1999:blog-3007054864987759910.post-90672477222765785502013-05-20T09:51:00.001-05:002013-05-20T09:51:56.208-05:00Submitting R jobs with BoscoThe Bosco team has been working on integrating with the <a href="http://www.r-project.org/">R</a> statistics processing language. We have chosen to modify the <a href="http://cran.r-project.org/web/packages/GridR/index.html">GridR</a> package in order to integrate with R.<br />
<br />
<h3>
How will the R user see Bosco?</h3>
<div>
The goal of the integration is to simplify the method of submitting processing, written in the R language, to remote clusters and grids. The expected steps for the integration are:</div>
<div>
<ol>
<li>Install Bosco</li>
<li>Install the Bosco'ified GridR package into your local R environment.</li>
</ol>
</div>
<div>
After installing the 2 pieces of software above, the user creates a R script, which includes the 'function' that is to be executed on the remote cluster. The user can send any data as input, lists, tables, an entire CSV file (already read into a R variable). The function output will be automatically imported into the environment when the remote job has completed.</div>
<div>
<br /></div>
<div>
Below is a demo of the GridR package working with Bosco to submit to a campus cluster here at Nebraska.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtvuey7QDLPR_QI4PMzBnJBLajgr13ERARa7WUc754P0AlpYzq60VYjaidJpcmc-fKtomgveoWeGfKj5cdrnnYFiIDfTl1nfhhfEBRIYA-74WmqZTGxnFlHQM9CY3T0ROeUmAIYGCoS7L8/s1600/Screen+Shot+2013-05-18+at+12.55.40+AM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="433" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtvuey7QDLPR_QI4PMzBnJBLajgr13ERARa7WUc754P0AlpYzq60VYjaidJpcmc-fKtomgveoWeGfKj5cdrnnYFiIDfTl1nfhhfEBRIYA-74WmqZTGxnFlHQM9CY3T0ROeUmAIYGCoS7L8/s640/Screen+Shot+2013-05-18+at+12.55.40+AM.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.rstudio.com/">RStudio</a> IDE showing demo of Bosco + GridR integration</td></tr>
</tbody></table>
<div>
The steps in the demo are:</div>
<div>
<ol>
<li>Load the GridR library</li>
<li>Create the function, in this case named simply 'a' that doubles the value of the argument.</li>
<li>Initialize the GridR integration to talk to Bosco</li>
<li>"Apply" the function. Run the function 'a', with the input 14, and write the result to the variable "x". Also, wait for the remote job to complete.</li>
<li>Finally, I printed out the value of x, which is 28, double the 14. </li>
</ol>
<div>
This is a very simple demo. You could imagine the function sent to the remote machine could parse the a CSV file, or more complex operations...</div>
</div>
<div>
<br /></div>
<div>
The Bosco team expects to have this integration done and in production by Mid-July for the <a href="http://161.67.142.97/congresos/useR-2013/%E2%80%8E">R users meeting</a>.</div>
<br />
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img alt="Bosco Download" src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png" style="border-width: 0;" />
</a>
</center>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com1tag:blogger.com,1999:blog-3007054864987759910.post-92213839568641867782013-04-02T14:38:00.003-05:002013-04-02T15:23:20.401-05:00Reprocessing CMS events with BoscoPrior to the <a href="http://home.web.cern.ch/about/updates/2013/02/long-shutdown-1-exciting-times-ahead">LHC long shutdown</a>, the CMS experiment increased the trigger rate of the detector, therefore increasing the data coming off the detector. The Tier-0 was unable to process all of the events coming off of the detector, therefore the events where only stored and not processed. After the run, the experiment wanted to process the backlog of events, but didn't have the computing power available to do it. So they turned to opportunistic computing and <a href="http://bosco.opensciencegrid.org/">Bosco</a>.<br />
<br />
The CMS collaborators at UCSD worked with the <a href="http://www.sdsc.edu/">San Diego Supercomputing Resource</a> to run the processing on the <a href="http://www.sdsc.edu/News%20Items/PR030512_gordon.html">Gordon</a> supercomputer. Gordon is an <a href="https://www.xsede.org/web/guest/sdsc-gordon">XSEDE resource</a> and does not include a traditional OSG Globus Gatekeeper. Also, we did not have root access to the cluster to install a gatekeeper. Therefore, Bosco was used to submit and manage the <a href="http://www.uscms.org/SoftwareComputing/Grid/WMS/glideinWMS/doc.prd/index.html">GlidienWMS</a> Condor glideins to the resource.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiEfVKGNDWFseJzsF-rHta1sQeuHaaj_SDeDSeQ_jsRoFWiSt7vC4MlLGVci8z8ohVLN1Jc6Ff3ykcfeEFrd5EjWDCns34jd4nthPpH_Y0roIJJf7JxELSLj2P4iQ2CK7gy3ObDW2QdUt5/s1600/sdsc_jobs.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiEfVKGNDWFseJzsF-rHta1sQeuHaaj_SDeDSeQ_jsRoFWiSt7vC4MlLGVci8z8ohVLN1Jc6Ff3ykcfeEFrd5EjWDCns34jd4nthPpH_Y0roIJJf7JxELSLj2P4iQ2CK7gy3ObDW2QdUt5/s640/sdsc_jobs.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 13px;">Running jobs at Gordon, the SDSC supercomputer</td></tr>
</tbody></table>
<br />
<br />
As you can see from the graph, we reached nearly 4,000 CMS processing jobs on Gordon. 4k cores is larger than most CMS Tier 2's, and as big as a European Tier-1. With Bosco, overnight, Gordon became one of the largest CMS clusters in the world.<br />
<br />
Full details will be written in a submitted paper to <a href="http://www.chep2013.org/">CHEP '13</a> in Amsterdam, and Bosco will be presented in a poster (and paper) as well. I hope to see you there!<br />
<br />
(If I got any details wrong about the CMS side of this run, please let me know. I have intimate knowledge of the Gordon side, but not so much the CMS side).<br />
<br />
<br />
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img alt="Bosco Download" src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png" style="border-width: 0;" />
</a>
</center>
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-45381007524059396382013-02-05T22:27:00.002-06:002013-02-09T13:51:53.758-06:00Using Bosco to submit to Amazon EC2A homework assignment for my storage class required running ~30 hours of benchmarks of the btrfs and ext4 filesystems. I thought this would be an excellent time to test Bosco's ability to submit to Amazon EC2 to parallelize the benchmarks.<br />
<br />
<h3>
Preparing Submission</h3>
<div>
In order to start instances on Amazon EC2, you first need to sign up. Go to <a href="https://aws.amazon.com/">https://aws.amazon.com/</a> and sign up in the top right. After you sign up, you will need the access and secret key. These can be found in the 'Security Credentials' from the account drop down box. They are in the 'Access Keys' tab under the 'Access Key ID' and 'Secret Access Key'. Write those values in 2 files, you will need them when you submit EC2 instances.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtYPUr5l_XlaCPFMMXJFiLQw45kjkqUzjnmfL2xQdsVIPgrLht4tVY1zJFRuuIgLGVzjgq9tIN-wyqbV6_x6LAvXQRIejRMCrHh0NliR0ng5p7FqE3PB2DCetxN9cSMQ8yfWg5wQ0Axntj/s1600/amazoncred.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="441" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtYPUr5l_XlaCPFMMXJFiLQw45kjkqUzjnmfL2xQdsVIPgrLht4tVY1zJFRuuIgLGVzjgq9tIN-wyqbV6_x6LAvXQRIejRMCrHh0NliR0ng5p7FqE3PB2DCetxN9cSMQ8yfWg5wQ0Axntj/s640/amazoncred.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Screenshot of Amazon Security credentials site</td></tr>
</tbody></table>
<div>
<br /></div>
<div>
<br /></div>
<div>
Next, you will need a script to run at startup of the Amazon instance. When the instance starts up, a service named <a href="https://help.ubuntu.com/community/CloudInit">CloudInit</a> also start on the instance. It will interpret the user data file as a shell script which can setup and start any other services you would like. My shell script is provided below.</div>
<div>
<div class="gistLoad" data-id="4719948" id="gist-4719948">
Loading ....</div>
</div>
<div>
<br /></div>
<div>
This shell script will install python-boto (python bindings for S3 storage and ec2) and git onto the instance. Next, it will download the filebenchrunner (Benchmark runner for the homework), and start it. Most people will probably want to shut down the instance after you are done with processing, in that case you can just add to the bottom a 'poweroff'. <br />
<br />
<h3>
Running the Instance</h3>
</div>
<div>
Running an Amazon instance is as easy as running a Bosco job. First, you must create a Bosco submit file. Below is the one I used:</div>
<div>
<div class="gistLoad" data-id="4720141" id="gist-4720141">
<br /></div>
<div>
<br />
Some important things to note. I specified ec2_spot_price, which is the amount I am willing to pay for my m1.medium instance to run per hour. I said $0.04 an hour, which is pretty low, but reasonable for a medium instance. You can find all of the current spot prices either in the AWS console, or on the EC2 <a href="https://aws.amazon.com/ec2/spot-instances/#7">website</a>. As you can see, the spot prices are much, much smaller than the on-demand price of an instance. For example, for the m1.medium instance, which has 1.7 GB of ram and 1 core, the spot price currently is $0.013 per hour. The on-demand price is $0.120 per hour. That's a 90% discount on a m1.medium. Of course, you should always read the <a href="https://aws.amazon.com/ec2/spot-instances/#4">downsides</a> of using a spot instance, such as it can be terminated at any time, without warning, by Amazon. For my benchmarks, I can always re-run benchmarks if my instance is terminated. I needed to run 10 - 10 minute benchmarks, therefore after every benchmark, I uploaded the resulting data to S3 immediately so I wouldn't lose any work if the instance was terminated.<br />
<br />
Also, I used the regular Amazon Linux AMI. They are listed on the Amazon <a href="https://aws.amazon.com/amazon-linux-ami/">website</a>. I could have very well used a CentOS, Ubuntu, or any other linux image for my instance. But, I prefer the official Linux AMI since it provides a very up to date OS which is very similar to the feel of a CentOS 6 instance. For example, it uses yum for repository management, and RPM's to install. And has versions (except for the kernel) similar to CentOS 6.<br />
<br />
I also added a special command, periodic_remove, in order to terminate the instance if something went wrong inside the instance. Sometimes yum can hang, or the instance may not start up properly. In those cases, amazon will not notify you of the problem, and Bosco will not be able to determine there is an issue. Since my benchmarks should not last longer than 100 minutes, I automatically remove the instance after 150 minutes (a little breathing room) of running.<br />
<br />
You may submit the instance with the normal 'condor_submit' command. The job will move to the <b>R</b>unning state when the instance has begun running.<br />
<br />
Once the instance has started, you may ssh into the instance by using the unique ssh key that Bosco generates for you. It is specified in the submit file as ec2_keypair_file. You also need the DNS name for the instance, which is available in the job's classad. <br />
<pre>condor_q -run</pre>
<br />
The command will output the hostname of the EC2 host. You may connect to the EC2 instance with the command, replacing the XXXX with the job number, and hostname with the address you get from the above command:<br />
<pre>$ ssh -i keyfile.XXXXX ec2-user@<span style="color: red;">hostname</span></pre>
<br />
<h3>
Summary</h3>
<div>
Pros of using Bosco to submit Amazon EC2 Jobs:</div>
<div>
<ul>
<li>Simple management of Amazon instance <b>from your workstation</b>.</li>
<li>Specify spot price right inside of the job description.</li>
<li>Ability to bootstrap the instance easily with user data scripts.</li>
<li>Ability to use HTCondor policies in order to manage the instances, such as periodic remove statement above.</li>
</ul>
Cons:</div>
<div>
<ul>
<li>The EC2 universe is only available on Linux builds of Bosco. You cannot manage EC2 instances on the Mac version of Bosco.</li>
<li>Amazon EC2 has hundreds and hundreds of features, Bosco only allows you to use the simple submit EC2 instances and spot pricing. You will not be able to use the vast majority when you are using Bosco to manage your instances. But if all you need is to run some processing, Bosco is great!</li>
</ul>
</div>
<br /></div>
<script src="https://raw.github.com/moski/gist-Blogger/master/public/gistLoader.js" type="text/javascript"></script></div>
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png"
alt="Bosco Download"
style="border-width: 0;" />
</a>
<br />
<a href="http://creativecommons.org/licenses/by/3.0/deed.en_US" rel="license"><img alt="Creative Commons License" src="http://i.creativecommons.org/l/by/3.0/88x31.png" style="border-width: 0;" /></a><br />
This work is licensed under a <a href="http://creativecommons.org/licenses/by/3.0/deed.en_US" rel="license">Creative Commons Attribution 3.0 Unported License</a>.
</center>Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com2tag:blogger.com,1999:blog-3007054864987759910.post-58544712945445708342013-01-16T14:02:00.001-06:002013-02-09T13:52:36.589-06:00Rendering with Bosco<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOac8m0n0z8Lq-ukGKMk82AB4qGsFI1La2jEKU6ih_xTpBrhHRF7KOVOv2LkXqEGNZS9mPmqmI05EFmBGNDDM0Gx7pocw7jE9JQWGXqMU6YIQ-ca-wo3xxh6m_0Oio2V5GAmCYZ_8qeYZK/s1600/0100_0050_00_LAY.189.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOac8m0n0z8Lq-ukGKMk82AB4qGsFI1La2jEKU6ih_xTpBrhHRF7KOVOv2LkXqEGNZS9mPmqmI05EFmBGNDDM0Gx7pocw7jE9JQWGXqMU6YIQ-ca-wo3xxh6m_0Oio2V5GAmCYZ_8qeYZK/s400/0100_0050_00_LAY.189.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Example image rendered using HCC's distributed renderer</td></tr>
</tbody></table>
<div>
<br /></div>
At the Nebraska Holland Computing Center, we take pride in <a href="https://en.wikipedia.org/wiki/Eating_your_own_dog_food">eating our own dog food</a>. So I want to highlight one of our uses of <a href="http://bosco.opensciencegrid.org/">Bosco</a> to enable transparent usage of our clusters.<br />
<div>
<br /></div>
<div>
HCC recently has made a push for enabling the non-traditional users of clusters. Sure it's easy to show how physics can benefit from a set of clusters, but what about the media arts? In this case we are working with a professor whose students render short movies using <a href="https://en.wikipedia.org/wiki/Autodesk_Maya">Maya</a>.</div>
<div>
<br /></div>
<div>
Maya can utilize another Autodesk product called Backburner, which enables rendering across multiple nodes. The challenge was to allow Backburner to operate in a shared cluster environment. The standard method to use Backburner expects nodes dedicated solely to running the Backburner Server daemons. This <a href="https://en.wikipedia.org/wiki/Master/slave_(technology)">Master-Worker</a> fits the traditional HTC model well, therefore we felt that Bosco would be a great fit to enable submission to our clusters.</div>
<div>
<br /></div>
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiypjchbzxugS4GsbulJfG5EsC4KXr_2zTe9qJN9h8hyNUedCmPhMe-AEbIdkJnSYw_2jjiCB92A1krhvyPwOd0eLQvcVP-AotFWtNygugM7ONQdFllmPaI0ijUq-qBENScHklWv248Xqa/s1600/setup_figure.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="492" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiypjchbzxugS4GsbulJfG5EsC4KXr_2zTe9qJN9h8hyNUedCmPhMe-AEbIdkJnSYw_2jjiCB92A1krhvyPwOd0eLQvcVP-AotFWtNygugM7ONQdFllmPaI0ijUq-qBENScHklWv248Xqa/s640/setup_figure.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Architecture Diagram of the BOSCO enabled render (Credit: Adam Caprez)</td></tr>
</tbody></table>
In the architecture shown in the image above, we run a central service, hcc-render that monitors the Backburner queue in order to submit the Backburner servers to the Bosco queue. Bosco then submits a glidein to Tusker, which in turn runs the Backburner server and renders the scene.<br />
<br />
In this case, since we are using Backburner server for the actual processing, the data is handled internally by Backburner. In practice this means that data is stored on the Tusker file system which is then mounted by the professors local machine that is in turn mounted by the clients.<br />
<br />
This is a classic example of deeply integrating HTC into the user's application. <br />
The user only needs to click the 'render' button <b>from within Maya on their workstation</b>, and we handle all the rest, automatically. This is only made possible because Maya has a sane back end renderer that is designed to run on Linux. This architecture may not work for all commercial applications.<br />
<br />
Even though the user isn't directly using Bosco, a primary goal of Bosco, this is an excellent use of Bosco to enable HTC workflows.</div>
<br />
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png"
alt="Bosco Download"
style="border-width: 0;"/>
</a>
</center>Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-19269029957503030012013-01-14T12:59:00.000-06:002013-02-09T17:49:44.937-06:00Bosco 1.1.1 ReleaseToday I am pleased to announce that Bosco version 1.1.1 has been released. This is a patch fix for the 1.1 to address 2 issues that where affecting users. The release is available on the <a href="http://bosco.opensciencegrid.org/download/">Bosco Download Page</a>.<br />
<br />
Release notes for the 1.1.1 release are available on the <a href="https://twiki.grid.iu.edu/bin/view/CampusGrids/Bosco1p1p1ReleaseNotes">OSG Twiki</a>.<br />
<br />
Bosco 1.1 was a major release with many new features. The 1.1 release notes are also available on the <a href="https://twiki.grid.iu.edu/bin/view/CampusGrids/Bosco1p1ReleaseNotes">OSG Twiki</a>.<br />
<br />
On behalf of the Bosco Team,<br />
Derek Weitzel<br />
Dan Fraser<br />
Marco Mambelli<br />
Jaime Frey<br />
Brooklin Gore<br />
Miha Ahronovitz <br />
<center>
<a href="http://bosco.opensciencegrid.org/download/">
<img src="https://raw.github.com/osg-bosco/bosco-download-images/master/images/download-orange.png"
alt="Bosco Download"
style="border-width: 0;"/>
</a>
</center>Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-24435358186518794132013-01-14T09:00:00.000-06:002013-01-14T12:20:33.737-06:00Improving Gratia's Web InterfaceOver the winter break, I worked on improving the interface that most users use for OSG accounting. When I returned from break, I worked with Ashu to integrate the changes with some recent changes he had made. The new interface runs on <a href="http://gratiaweb-itb.grid.iu.edu/gratia/">gratiaweb-itb</a>. The source for the new web page is hosted on <a href="https://github.com/ashuguru/OSG-Measurements-Metrics">github</a>.<br />
<br />
The first thing users will notice is the newly designed interface:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJOPTKiCH4y3TrkPx1hWko3ov7F13bGCQYTk5Huh9RXhe744uQuLoNlveCaqO4k0B2LuPtxkre_7zxdbEw4hCMf8Wj6DqFHDG3JT4yYHJ2qgrvWbKYyxMorREy36RCO32v9CrxNYDB6f9z/s1600/Screen+Shot+2013-01-12+at+9.31.25+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJOPTKiCH4y3TrkPx1hWko3ov7F13bGCQYTk5Huh9RXhe744uQuLoNlveCaqO4k0B2LuPtxkre_7zxdbEw4hCMf8Wj6DqFHDG3JT4yYHJ2qgrvWbKYyxMorREy36RCO32v9CrxNYDB6f9z/s640/Screen+Shot+2013-01-12+at+9.31.25+PM.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New OSG Accounting Interface</td></tr>
</tbody></table>
The updated interface brings the style of the website inline with that of <a href="http://myosg.grid.iu.edu/about">MyOSG</a> and <a href="https://oim.grid.iu.edu/oim/home">OIM</a> (or, close). The design stayed close the the original, but the menu on the right has changed significantly. First is just the style of the menu. But also we added a new category, Campus and Pilot View. <br />
<br />
In the Campus and Pilot View, we have some new graphs that show usage by GlideinWMS, Campus users, XSEDE users, and in the future, <a href="http://bosco.opensciencegrid.org/">Bosco</a> users.<br />
<br />
Lets run through a quick example. In this example, lets assume I'm a VO manager and want to see where my VO is running, how many hours, and who is running. <br />
<br />
<ol>
<li>Select the <a href="http://gratiaweb-itb.grid.iu.edu/gratia/pilot">Pilot & Campus Accounting</a> link.</li>
<li>Scroll to the bottom, to the Refine View.</li>
<li>Enter your VO name into the VO text box and hit enter.</li>
</ol>
<div>
This will pull up the custom page that shows usage for only your VO. For example, if I look at the osg VO:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAwlr8V_1EYEOPZ4Kjj_AFkY4Pf_qWqcH3074Ah82jB5JpW06nSx86hKnBiTKjPmo89btyLOEhARQ_YukDZERXOL4koFp3QgIP6r1V9m4ubveWZiidFx5u1pRjM8LsvJtYb2k9cMY1fzFv/s1600/Screen+Shot+2013-01-12+at+10.22.22+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAwlr8V_1EYEOPZ4Kjj_AFkY4Pf_qWqcH3074Ah82jB5JpW06nSx86hKnBiTKjPmo89btyLOEhARQ_YukDZERXOL4koFp3QgIP6r1V9m4ubveWZiidFx5u1pRjM8LsvJtYb2k9cMY1fzFv/s640/Screen+Shot+2013-01-12+at+10.22.22+PM.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Usage by the OSG VO.</td></tr>
</tbody></table>
<div>
You can see from the graphs that the OSG VO has used ~80,000 CPU hours a day on the OSG. Also, they are running at over 20 sites. The sites at the bottom of the graph are listed in order of total hours (I am happy to see Nebraska resources as #3, #6, and #9). </div>
<div>
<br /></div>
<div>
You can also see from the graph that usage at sites depends on the day. Some days they get significant usage at the MWT2 (UChicago and IU), and some days they run a lot at Nebraska.</div>
<div>
<br /></div>
<div>
The new usage graphs are intended to help users, administrators, and VO managers view their usage. I hope you find them as useful as we have in the past.</div>
<div>
<br /></div>
<div>
We hope that webpage is an improvement. If there are any comments on further improvements, we are interested in your feedback.</div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-15124167982364673412012-11-29T00:33:00.000-06:002012-11-29T12:07:06.006-06:00BOSCO v1.1 Features: Multi-Cluster SupportThis is the forth in the in the series of features in the 1.1 release of BOSCO. The previous posts have focused on <a href="http://derekweitzel.blogspot.com/2012/10/bosco-v11-features-ssh-file-transfer.html">SSH File Transfer</a>, <a href="http://derekweitzel.blogspot.com/2012/11/bosco-v11-features-single-port-usage.html">Single Port Usage</a>, and <a href="http://derekweitzel.blogspot.com/2012/11/bosco-v11-features-multi-os-support.html">Multi-OS Support</a>. All of these features are new in the 1.1 release. But now I want to talk about a feature that was in the previous 1.0 release, but is important enough to discuss again, Multi-Cluster Support. This feature is very technically challenging, so I will start with why you care about Multi-Cluster in BOSCO.<br />
<br />
<h3>
Why do you care about Multi-Cluster?</h3>
<div>
On a typical campus, the each department may have it's own cluster for their use. Physics may have a cluster, Computer Science has one, and Chemistry may have another. Or a computing center may have multiple clusters reflecting multiple generations of hardware. In either of these cases, users have to pick which cluster to submit jobs to, rather than submitting to which ever has the most free cores.<br />
<br />
You don't care what cluster you run on. You don't care how to submit jobs to the PBS Chemistry cluster or the SGE Computer Science cluster. In addition, who wants to learn two different cluster submission methods. You only care about finishing your research.<br />
<br />
BOSCO can unify the clusters by overlaying each cluster with an on demand Condor cluster. That way, you only learn the Condor submission method. The Condor job you submit to BOSCO will then be run at whichever cluster has the first free cores for you to run on.<br />
<br /></div>
<h3>
What Is Multi-Cluster?</h3>
<div>
In BOSCO, Multi-Cluster is the feature that allows for submission to multiple clusters with a single submit file. A user can submit a regular file, such as:<br />
<pre>universe = vanilla
output = stdout.out
error = stderr.err
Executable = /bin/echo
arguments = hello
log = job.log
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
queue 1
</pre>
<br /></div>
<div>
When the job is submitted, BOSCO will submit glideins to each of the clusters that are configured in BOSCO. The glideins will start on the worker nodes of the remote clusters and join the local pool at the submission host, creating an on-demand Condor pool. The jobs will then run on the remote worker nodes through the glideins. This may be best illustrated by a picture I made for my thesis:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgizqi0L_E4YK8IubvzzRRDcVdcWc8btrfFd-D05iKV1-eFXHMaQzspCvb7mMx90sU0rA1o1h4BQ66blbNblBc9oExhjJYiLxK32-JlR2Yw64bs6vxdxPTHFGs_PIswMxGMthetHn-jtNK9/s1600/BOSCOPilotLaunch.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgizqi0L_E4YK8IubvzzRRDcVdcWc8btrfFd-D05iKV1-eFXHMaQzspCvb7mMx90sU0rA1o1h4BQ66blbNblBc9oExhjJYiLxK32-JlR2Yw64bs6vxdxPTHFGs_PIswMxGMthetHn-jtNK9/s640/BOSCOPilotLaunch.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Overview of the BOSCO job submission to a PBS cluster</td></tr>
</tbody></table>
<br />
In the diagram, first BOSCO will submit the glidein to the PBS cluster. Second, PBS will schedule and start the glidein on a worker node. Finally, the glidein will report to the BOSCO submit host and will start running user jobs.<br />
<br />
The Multi-Cluster feature will continue to be a integral part of the BOSCO.<br />
<br />
The Beta release of BOSCO is due out next week (fingers crossed!). Be watching this blog, and the <a href="http://bosco.opensciencegrid.org/">BOSCO website</a> for more news.<br />
<br />
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-85161826729525252282012-11-06T16:11:00.005-06:002012-11-06T16:38:40.150-06:00BOSCO v1.1 Features: Multi-OS SupportThis is part 3 of my ongoing series describing new features in BOSCO v1.1. Part 1 covered <a href="http://derekweitzel.blogspot.com/2012/10/bosco-v11-features-ssh-file-transfer.html">file transfer over SSH</a>, Part 2 covered <a href="http://derekweitzel.blogspot.com/2012/11/bosco-v11-features-single-port-usage.html">single port usage</a>. This post will cover the Multi-OS Support in BOSCO.<br />
<br />
<h3>
What is it?</h3>
<div>
The Multi-OS feature is intended to allow users to submit to clusters that may not be the same operating system as the submit host. This is especially useful when users are running a submit host on their personal computer that is running Debian, while the supercomputer in the next building is running Red Hat 6, a common occurrence.</div>
<div>
<br /></div>
<div>
The Multi-OS components follow a basic process in order to operate:</div>
<div>
<ol>
<li>Detect the remote operating system with the <span style="font-family: Courier New, Courier, monospace;">findplatform</span> script.</li>
<li>Download (from the cloud?) the appropriate bosco version for the platform.</li>
<li>Transfer the files needed on the remote cluster. This includes grabbing the libraries and binaries for the campus factory's glideins. The glidein creation takes the most time of this process as it needs to compress the libraries and binaries before transferring.</li>
<li>When BOSCO detects jobs idle on the submit host, it will start glideins appropriate for the platform to service the jobs.</li>
</ol>
<div>
The Multi-OS support required modification of the cluster addition and adding both a <span style="font-family: Courier New, Courier, monospace;">findplatform</span> script and a <span style="font-family: Courier New, Courier, monospace;">glidein_creation</span> script. </div>
</div>
<div>
<br /></div>
<h3>
Why do I care?</h3>
<div>
It is becoming increasingly common that what users run on their machines and what is run on supercomputers are different. When this is true, it is difficult to install software from one onto the other. The Multi-OS feature will greatly simplify the installation of BOSCO on clusters.</div>
<div>
<br /></div>
<div>
Our goal with the Multi-OS support is that the users may not know it is even working. The users just say: "I want to run on this cluster", and BOSCO makes it happen. No matter what operating system is running on the remote cluster.</div>
<div>
<br /></div>
<div>
One of my tests simulated a possible user scenario I was running a updated RHEL 6 machine which I installed BOSCO. I wanted to submit jobs to a RHEL 5 cluster located in our datacenter. If I simply copied over the bosco install from the RHEL 6 submit host, none of the binaries would work. But instead, I used the <span style="font-family: Courier New, Courier, monospace;">bosco_cluster -a</span> to add the RHEL 5 cluster, and jobs ran seamlessly from the RHEL 6 machine to the RHEL 5. </div>
<br />
The Multi-OS support is available in the latest alphas available on the bosco <a href="http://bosco.opensciencegrid.org/download/">download</a> page.<br />
<br />Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-23555222355000043112012-11-01T15:59:00.000-05:002012-11-06T16:38:50.521-06:00BOSCO v1.1 Features: Single Port UsageWelcome to part 2 of my ongoing series of v1.1 features for BOSCO. Part 1 was on <a href="http://derekweitzel.blogspot.com/2012/10/bosco-v11-features-ssh-file-transfer.html">SSH File Transfer</a>.<br />
<br />
This time, I'll talk about a new feature that we didn't planned on implementing at first, using only a single port for all communication. After a small investigation, it was discovered that using a single port is very simple, and with no interruption to other components. I talked briefly about it in a <a href="http://derekweitzel.blogspot.com/2012/09/flocking-to-osg-behind-restrictive.html">previous post</a>.<br />
<br />
<h3>
What is it?</h3>
<div>
In 1.0 of BOSCO, the submit host needed a lot of ports open for connections originating from the remote clusters. This was caused by 2 mechanisms:</div>
<div>
<ol>
<li>File transfer from the BOSCO submit host to the cluster login node before issuing the local submit call (qsub, condor_submit...). This opens ports on the submit host because the cluster would call out to the submit host to initiate transfers.</li>
<li>Connections for control, status, and workflow management between the cluster worker nodes and BOSCO submit host. This is the Campus Factory, which gives BOSCO the traditional Condor look and feel.</li>
</ol>
In order for BOSCO to function, the submit host needs a large swath of ports in order to operate correctly. Also, as you scale, you will need even more ports open.</div>
<div>
<br /></div>
<div>
The file transfers from the submit host to the login node are now being transferred using SSH, see my <a href="http://derekweitzel.blogspot.com/2012/10/bosco-v11-features-ssh-file-transfer.html">previous post</a>.</div>
<div>
<br /></div>
<div>
With the new feature of single port usage, all control, status, and workflow management connections are routed through HTCondor's <a href="http://research.cs.wisc.edu/condor/manual/v7.9/3_7Networking_includes.html#SECTION00472000000000000000">share_port_daemon</a> on port 11000 (which is hardcoded, but I picked at random).</div>
<div>
<br /></div>
<h3>
Why should I care?</h3>
<div>
Limiting BOSCO to using only 1 incoming port is very useful for users on systems not managed by them. The node will only need 1 port open in order to run BOSCO, 11000. If the system has a firewall, you only have to request port 11000 be opened, rather than huge swaths. If you manage the system, then you will be happy that only 1 port needs to be opened in order to allow BOSCO submissions.</div>
<div>
<br /></div>
<div>
Administrators will like this feature as it is more in line with other applications that they may run. For example, httpd only requires 1 port, 80. Now BOSCO is in the same realm, only requiring 1 port, 11000.</div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-32372811538095978702012-10-29T14:30:00.002-05:002012-11-06T16:39:02.268-06:00BOSCO v1.1 Features: SSH File TransferI am hoping to write about a few of the new features of BOSCO 1.1 before it comes out in December or January. This is part 1 of that series:<br />
<br />
<h3>
BOSCO v1.1 Feature: SSH File Transfer</h3>
<div>
<b>What is it?</b></div>
<div>
SSH File Transfer feature improves the method of staging input and output files to the remote cluster. In 1.0, files are transferred by starting a daemon on the remote cluster that connects back to the submit host over a random port. This required a lot of open ports on the submit host. </div>
<div>
<br /></div>
<div>
The new SSH File Transfer will limit the number of ports required on the submit host. BOSCO will now transfer files over a port that is forwarded over the SSH connection that BOSCO maintains with the remote cluster. The transfers are inherently secure as they are over the SSH connection, as well as they are authenticated by the Condor daemons on either end of the connection (remote cluster and submit host). </div>
<div>
<br /></div>
<div>
This fits into the BOSCO team's goal of lowering the amount of ports used by Condor. Our eventual goal is using the Shared Port Daemon to limit the required ports to 1 for BOSCO on the submit host.</div>
<div>
<br /></div>
<div>
<b>Why should I care?</b></div>
<div>
This will greatly reduce the number of ports required if you are only using the <span style="font-family: Courier New, Courier, monospace;">universe=grid</span><span style="font-family: inherit;"> method of submitting jobs. In fact, it will reduce the open ports on the submit host to 0. That means no more configuring firewalls for BOSCO (no campus factory support, see below). Additionally, there is no new configuration required for this feature, it 'just works' (famous last words?)</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">The Campus Factory, which adds features such as fault tolerant Condor file transfer and transparent multi-cluster support, still requires multiple open ports in the firewall. Additional effort will be required to change the Campus Factory configuration and daemons to support the single port. I hope that a single port will be all that is needed for v1.2. </span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<h3>
<span style="font-family: inherit;">What's Next</span></h3>
<div>
<span style="font-family: inherit;">Over the next couple weeks, I hope to write more about upcoming features such as:</span></div>
<div>
<ul>
<li>Multi-Platform support (ie cluster and submit host are different platforms)</li>
<li>Mac OSX Support</li>
<li>Improved Multi-User support</li>
</ul>
</div>
<div>
<span style="font-family: inherit;"><br /></span></div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-17596603803616340312012-10-18T10:46:00.001-05:002012-10-18T10:46:09.784-05:00The Island Grid: Puerto RicoIn the <a href="http://derekweitzel.blogspot.com/2012/10/building-campus-grid-on-island.html">last post</a>, I discussed my trip to Puerto Rico. Now that my trip is over, and I am back in Lincoln, I would like to share some of the successes of the trip. I had a great time exploring old San Juan. I have <a href="https://plus.google.com/photos/100333871316337489574/albums/5799337435650454001?authkey=CN6x_9GI1OKrcA">pictures</a> as well.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-rOaHnTp-RDY/UHzTARfukVI/AAAAAAAABwY/OWJpEkkSdYs/s1600/IMG_0302.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-rOaHnTp-RDY/UHzTARfukVI/AAAAAAAABwY/OWJpEkkSdYs/s400/IMG_0302.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Presenting the Holland Computing Center grid</td></tr>
</tbody></table>
<br />
<h2>
Creating a new User</h2>
But on a more relevant note, the Campus Grid in Puerto Rico gained a new user while I was there. I always feel that computing infrastructure is best built by user's demand. That has certainly been the case at HCC, where we run between 95%+ utilization on our HPC clusters. I met with Steve Massey to find how the Island Grid can help him.<br />
<br />
<a href="http://biology.uprrp.edu/people/view_employee.php?id=201">Steve Massey</a> is a bio-informatician at the University of Puerto Rico -- Rio Pedres. His work is an ideal fit for High Throughput Computing. His processing follows the model of using the same executable against many, many protein pdb files. We talked for a while on Tuesday, both before and after the power was cut to the UPR campus, about how we can enable this work onto the UPR campus grid, flocking to UNL, and finally to the OSG. While I was in PR, I worked with Steve to run one of his workflows on the OSG.<br />
<br />
I'm not going to pretend to know what is really happening with the workflow, but it takes as input a set of DNA sequences that where pre-calculated (I believe somewhat random) and a pdb file, and calculates the robustness using an external application, Scwrl4. The output is a robustness file that lists the robustness for the protein with the DNA sequences.<br />
<br />
I was able to run this workflow on the OSG using Nebraska's GlideinWMS interface. I created a submit.sh script that wrote out a simple Condor submit script, and wrapper.sh, that configured the environment on the worker node. Both of the scripts are available on <a href="https://gist.github.com/3912441">github</a>. Together, these two components create the workflow. <br />
<br />
There is still work to be done. The executable that Steve wrote does not properly detect the length of the strand of the amino acid, and therefore is not able to properly calculate robustness and/or send to Scwrl4.<br />
<br />
Also, there is another workflow that Steve would like to run. I hope to continue to work with him to enable these workflows.<br />
<br />
<h2>
Onto the Grid!</h2>
<div>
In addition to talking to Steve about creating workflows, I also discussed how to integrate his small cluster into the Island Grid in Puerto Rico. The primary difficulty with adding his cluster is the university's very restrictive firewall. Therefore, we will be unable to run a central submission host on his cluster, and flocking is out. But, I believe between Jose and myself, we have a solution. Either he can run BOSCO and submit through SSH to Steve's SGE cluster, or install Condor worker nodes on Steve's cluster to report to the primary campus cluster, nanobio.</div>
<div>
<br /></div>
<div>
I also talked with a Computer Science researcher to discuss putting his Mac Mini cluster on the Grid. He is running Linux on the Mac Mini's, therefore it would be a good fit for the grid. The frontend node to the Mac Mini cluster is not running linux, therefore BOSCO will not work as it needs a consistent platform on the frontend node and the worker nodes. Instead we will have to install Condor worker nodes on the nodes themselves in order to access the Mac Mini's.</div>
<div>
<br /></div>
<h2>
Conclusions</h2>
<div>
I believe that Puerto Rico has a lot to benefit from creating an Island (Campus) Grid. They also have a very energetic administrator, Jose, who can spearhead the implementation, with close collaboration with myself and the campus grids team.</div>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-27147630474825194032012-10-15T13:43:00.002-05:002012-10-15T13:43:21.791-05:00Building a Campus Grid on a IslandHello everyone from San Juan, Puerto Rico. <br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRC7DErdLTYlPiRatY8L0XByLFX32yJnCFIzZ-994qWQnEFyc76BCwjwLykL0_-4cS6uyPhNh0KZSj2Adi4SRzse-gghsJv4lAXWWF73C0_myco7fdRVQfBvmADAbR-QNW8Ip1ab-IzkXS/s1600/2012-10-14+11.21.15+HDR.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRC7DErdLTYlPiRatY8L0XByLFX32yJnCFIzZ-994qWQnEFyc76BCwjwLykL0_-4cS6uyPhNh0KZSj2Adi4SRzse-gghsJv4lAXWWF73C0_myco7fdRVQfBvmADAbR-QNW8Ip1ab-IzkXS/s400/2012-10-14+11.21.15+HDR.jpg" width="300" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Hanging Out</td></tr>
</tbody></table>
I joined my advisor, David, to the Track 2 external advisory board meeting. My goals while here are:<br />
<br />
<ul>
<li>Work with Steve Massey to enable his workflow to run on the UPR campus grid.</li>
<li>Enable HPCf to flock with other local Condor clusters (Steve's and CS's).</li>
<li>Continue to work with HPCf to enable gratia accounting on their resources.</li>
<li>Enable HPCf to Nebraska flocking.</li>
</ul>
Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-73265213324087256142012-06-29T11:01:00.004-05:002012-06-29T11:01:36.977-05:00CMS with the Campus Factory<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The Campus Factory is usually used by small research groups to expand their available resources to those on the campus. Of course, that's not always easy for larger VO's, who tend to have more complicated software setups. This is where the combination of <a href="http://nd.edu/~ccl/software/parrot/">Parrot </a>and <a href="http://cernvm.cern.ch/portal/">CernVM-FS</a> comes in.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="http://cernvm.cern.ch/portal/img/fs-logo.png" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" src="http://cernvm.cern.ch/portal/img/fs-logo.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://cernvm.cern.ch/portal/filesystem">Source: http://cernvm.cern.ch/portal/filesystem</a></td></tr>
</tbody></table>
<br />
<div>
CernVM-FS is a HTTP based file system that serves many CERN based VOs software repositories. In our case, we used a CernVM-FS server hosted at the University of Wisconsin - Madison (<a href="http://www.hep.wisc.edu/cms/comp/ops/cvmfs.html">Docs</a>)</div>
<div>
<br /></div>
<div>
Parrot is a program that will capture reads and writes from arbitrary executables and redirect them to remote resources. For our use, we will redirect reads from the local file system to reads from the CernVM-FS server at UW.</div>
<div>
<br /></div>
<div>
Our T3, as usual, is over subscribed. Sending our T3 jobs out onto the grid, much like <a href="http://derekweitzel.blogspot.com/2011/04/overflowing-teir-2.html">overflowing Tier 2 jobs</a>, would significantly decrease the time to completion for our CMS users. But, our campus grid does not have CMS software available everywhere, therefore we must export the software to the jobs. For this, we use Parrot and CernVM-FS.</div>
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjD2Mxckbe-UIguAOmq9dbI0fIBI6czy9aNsT1A5aRJIx2Sm317Y2EVNoU_MjCCCtQX6hNaDk_i_j6v5ZV7fSVnWExwj5xm1W_HdEG0MW5SA9HszxndrsCgsRfbGzVRAkbYlDVwFgP4cSF3/s1600/CHEPCGF.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjD2Mxckbe-UIguAOmq9dbI0fIBI6czy9aNsT1A5aRJIx2Sm317Y2EVNoU_MjCCCtQX6hNaDk_i_j6v5ZV7fSVnWExwj5xm1W_HdEG0MW5SA9HszxndrsCgsRfbGzVRAkbYlDVwFgP4cSF3/s640/CHEPCGF.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pilot submission of BOSCO</td></tr>
</tbody></table>
<br />
<br />
The BOSCO system is depicted in the above graphic. First the user submit their job to their local Condor. This instance of Condor could be tied to local resources that also can run their jobs, but for this picture, we only show the BOSCO resources. The Factory periodically queries the user's Condor, and submits Pilot jobs to run the user's jobs. Once the pilots start on the remote system, they begin executing the users' jobs. The user does not have to specify any special requirements, nor use any special commands for this system to work.<br />
<br />
We used BOSCO to flock jobs from our T3 to our other campus resources. This process required no user interaction. Matter of fact, the user had no idea that her jobs where not running on the T3. This transparent interaction with the user is the primary goal of the Campus Factory design, and was clear in this experiment.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYYiwzCddk_SPMWlRrQFoRGGIEm_vZOtwnszHbqEIActrbznOXyIhZoLAgcPdb_btrG7FhQND-HXUZMkxYw3k8BIbJKVZxcynDfjVgA45wPfNkmL5d75-8fS3lFaTmuZN-r2eMNyNRqbNK/s1600/CHEPTier-3Campus.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYYiwzCddk_SPMWlRrQFoRGGIEm_vZOtwnszHbqEIActrbznOXyIhZoLAgcPdb_btrG7FhQND-HXUZMkxYw3k8BIbJKVZxcynDfjVgA45wPfNkmL5d75-8fS3lFaTmuZN-r2eMNyNRqbNK/s400/CHEPTier-3Campus.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Tier-3 Connection to the UNL Campus Grid</td></tr>
</tbody></table>
<br />
<div>
We hope to make this a production service in the future. In the meantime, this is being used as a prototype for what other Campuses can do with BOSCO.<br />
<br /></div>
</div>
<div>
<span style="background-color: white;">Acknowledgments</span>: Dan Bradley and the ccTools team for the CernVM-FS integration with parrot. The AAA project for the file infrastructure to enable transparent data access. And Helena Malbouisson for allowing me to play with her jobs, sending them to other resources.</div>
<div>
Modifications to campus factory configs can be found on <a href="https://github.com/djw8605/CVMFS-CampusFactory">github</a>.</div>Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com1tag:blogger.com,1999:blog-3007054864987759910.post-91712944339214243762012-04-16T16:29:00.000-05:002012-04-16T16:36:34.475-05:00BOSCO + Campus Factory<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVMbnriAg8a0vT-rJEzktVi8Z_MAF3aQeOBymtmzK6kdWszC6HvDoFVotaDq1Pmrh7Tkca0LpI1qh5XxMRhyphenhyphenAWN6eoVPbeCTT1qB9fLQgK4WSmKMUdy5NA7xYIYy3bNPL5ALhRXySZYU2n/s1600/photo.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVMbnriAg8a0vT-rJEzktVi8Z_MAF3aQeOBymtmzK6kdWszC6HvDoFVotaDq1Pmrh7Tkca0LpI1qh5XxMRhyphenhyphenAWN6eoVPbeCTT1qB9fLQgK4WSmKMUdy5NA7xYIYy3bNPL5ALhRXySZYU2n/s400/photo.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Checklist while implementing CF + Bosco integration</td></tr>
</tbody></table>
<br />
For the last several months, the<a href="https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome"> campus infrastructure</a> team has worked on software that will help users create a larger, more inclusive campus grid. The goal largely has been to make the software easier to install and expand.<br />
<br />
Integrating the <a href="https://twiki.grid.iu.edu/bin/view/Documentation/CampusFactoryInstall">Campus Factory</a> (which is already used on many campuses) with <a href="https://twiki.grid.iu.edu/bin/view/CampusGrids/BoSCO">Bosco</a> has been a key goal for this effort. Last week, I finally integrated the two (<a href="https://twiki.grid.iu.edu/bin/view/Documentation/CampusFactoryInstallRC">Beta Install Doc</a>). This will have many benefits for the user over both the current Campus Factory and current Bosco.<br />
<br />
<table id="box-table-b" style="width: 600px;">
<tbody>
<tr>
<th>Feature
</th>
<th>Campus Factory
</th>
<th>Bosco v0
</th>
<th>Campus Factory + Bosco
</th>
</tr>
<tr>
<td>Installation
</td>
<td>Large installation/configuration instructions. Install Condor and campus factory on every cluster.
</td>
<td colspan="2">Install on a central submit node. Configuration is handled automatically.
</td>
</tr>
<tr>
<td>Adding Resources
</td>
<td>Install Condor and the campus factory on every cluster. Configure to link to other submit and campus factory clusters.
</td>
<td colspan="2">Run the command <br />
<pre>bosco_cluster -add</pre>
Installation and configuration handled auto-magically.
</td>
</tr>
<tr>
<td>File Transfer
</td>
<td>Using Condor file transfer, can transfer input and output.
</td>
<td>Manually with scp by the user. No stderr or stdout from job.
</td>
<td>Using Condor file transfer, can transfer input and output.
</td>
</tr>
<tr>
<td>Job State
</td>
<td>Accurate user job state.
</td>
<td>Delayed user job state. No exit codes from the user jobs.
</td>
<td>Accurate user job state.
</td></tr>
</tbody></table>
As you can see from the above table, the combined Campus Factory + Bosco takes the best from both technologies.Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0tag:blogger.com,1999:blog-3007054864987759910.post-65775105689444725282012-03-27T17:24:00.001-05:002012-03-27T17:36:34.147-05:00OSG AHM 2012<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9YdFcTDihdCWw9_f3njgZucpz4jDqZsbmK0YrgdxcJ5pXEQ3jouwGJUUMqOy_yJ5ARFV_vAVeO-5eMo-p-4FhZYNIQ9e-15sKbhnPX2l5dl0CVWubivplvOXN_3XOpMjK1M17eSor3-_6/s1600/welcomesign.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="142" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9YdFcTDihdCWw9_f3njgZucpz4jDqZsbmK0YrgdxcJ5pXEQ3jouwGJUUMqOy_yJ5ARFV_vAVeO-5eMo-p-4FhZYNIQ9e-15sKbhnPX2l5dl0CVWubivplvOXN_3XOpMjK1M17eSor3-_6/s400/welcomesign.png" width="400" /></a></div>
<br />
<br />
This years <a href="https://indico.fnal.gov/conferenceDisplay.py?confId=5109">all hands</a> meeting was a great success! There where a few sessions that I really enjoyed.<br />
<br />
<a href="http://flic.kr/s/aHsjyJGT8G">AHM Pictures</a><br />
<br />
<span style="font-size: large;"><b>
Campus Caucus</b></span><br />
There where many user engagement people there. I believe that we reached a consensus that there isn't much an engagement community. <br />
<br />
For example, Wisconsin has a great method for distributing and running MATLAB and R application on the OSG, but there has been no knowledge transfer to other engagement folks. I know a few UNL users that have wanted to run MATLAB on our resources. If we could move a HCC MATLAB workflow to the Grid, I believe that would be a great success.<br />
<br />
I completely agree that there is no Engagement 'community'. But I think that's true of most of the OSG. Though, there have been recently many improvements. <br />
<br />
<ul>
<li>I think the centralized Jira has helped tremendously. It's very easy to see what other people have been working on and even the general direction of progress. Though this only works for OSG 'employees' and OSG projects.</li>
<li>The OSG blogs have been successful for the technology group to explain what they are working on. Though, I wish they had shorter and more often posts.</li>
</ul>
I hope that the blogs can be a way to spread the OSG Engagement activity. It's also a great way to point to code and work that is being done. Also, blog posts shouldn't be limited to things that the author is doing, but could point to what other people are doing. For example, I knew nothing about the Rosetta people at Wisconsin until my poster was setup next to theirs and was able to have a conversation. It would have been great to see some information that of what they where doing outside of the once a year meeting.<br />
<br />
<br />
<span style="font-size: large;"><b>
Science on the OSG</b></span><br />
<div>
<a href="https://indico.fnal.gov/getFile.py/access?contribId=12&sessionId=5&resId=0&materialId=slides&confId=5109">Talk</a></div>
<div>
<br /></div>
<div>
I thought this talk by Frank was great. I felt he had the same feeling that we where all feeling, that Protein processing was becoming a very large user of the OSG. We've seen this at HCC with both CPASS, CS-Rosetta, and Autodock. </div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy4dQIItMVph6qObPlOLOz7RgiQRprE-bGtnofi_nnfmVkDQgRf9_Mf9KUJe1Y2we5_5q_2DC-qLuCzaXhssW99B55lL6afcscRjLd6QNfbbwSzMlTgn9halUbCE9kCn7omAptjqAnQFWn/s1600/Screen+Shot+2012-03-27+at+3.47.43+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy4dQIItMVph6qObPlOLOz7RgiQRprE-bGtnofi_nnfmVkDQgRf9_Mf9KUJe1Y2we5_5q_2DC-qLuCzaXhssW99B55lL6afcscRjLd6QNfbbwSzMlTgn9halUbCE9kCn7omAptjqAnQFWn/s400/Screen+Shot+2012-03-27+at+3.47.43+PM.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Walltime usage for non-HEP</td></tr>
</tbody></table>
<div>
Frank also pointed out a graph of usage. At the end of the graph, there seems to be a plateau. Possibly we are hitting opportunistic resource limits?</div>
<div>
<br />
Here's an updated usage graph (<a href="http://gratiaweb.grid.iu.edu/gratia/xml/osg_wall_hours?endtime=2012-03-27+23%3A59%3A59&span=604800&facility=.*&probe=.*&resource-type=^Batch%24&vo=.*&role=.*&user=.*&starttime=2007-06-01+00%3A00%3A00&exclude-facility=NONE|Generic|Obsolete&exclude-user=NONE&includeSuccess=true&exclude-vo=unknown|other|vo|atlas|cdf|dzero|cms|Unknown&includeFailed=true&exclude-role=NONE">source</a>):<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIRYY3DTYQruk3hREyaVPM9W3RLbe6WuD8muR_JCYhNBpgNLaiY0Y5ElbrcDZU2QBgzkKwqNyfTNTX24QEKkywtl9hGl3P5A9vIdL96LO-KspiZU0Q58bBYvmc1-_CCQgT976lI-uafEc8/s1600/osg_wall_hours.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIRYY3DTYQruk3hREyaVPM9W3RLbe6WuD8muR_JCYhNBpgNLaiY0Y5ElbrcDZU2QBgzkKwqNyfTNTX24QEKkywtl9hGl3P5A9vIdL96LO-KspiZU0Q58bBYvmc1-_CCQgT976lI-uafEc8/s400/osg_wall_hours.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Walltime usage for non-HEP updated</td></tr>
</tbody></table>
The thing to note is the explosive growth of GLOW VO. Their usage has increased dramatically recently.<br />
<br />
<br /></div>
<span style="font-size: large;"><b>
OSG in 2017</b></span><br />
<div>
<a href="https://indico.fnal.gov/getFile.py/access?contribId=19&sessionId=5&resId=0&materialId=slides&confId=5109">Talk</a></div>
<div>
<br /></div>
<div>
I really liked to see what people thought the OSG would look like in 2017.</div>
<div>
<br /></div>
<div>
Chander's prediction that people will come to us to use the OSG. I believe this will take a critical mass of users. I think we have a good product to sell, we just need publicity. </div>
<div>
<br />
Chander's comment on data is also important. But I believe the problem with data isn't necessarily storage, but it's access to the data. Take for example Dropbox. For free, they offer very little storage. The main advantage is that it's accessible from anywhere, laptop, desktop, iphone, web... I think a uniform data access method can get us a lot further than distributed storage.</div>
<div>
<br />
Alain's prediction that we will be using more community software. This will take a large effort to be part of the distribution's community. I foresee us contributing packages, patches, and effort to Fedora EPEL and possibly Ubuntu. I think we are making great strides with the packaging, and would like to continue injecting us into the Fedora community.</div>
<div>
<br />
<br /></div>
<span style="font-size: large;"><b>
Nebraska Campus Infrastructure</b></span><br />
<div>
<a href="https://indico.fnal.gov/getFile.py/access?contribId=48&sessionId=5&resId=0&materialId=slides&confId=5109">Talk</a></div>
<div>
<br /></div>
<div>
Of course, my talk is worth looking at.</div>
<div>
<br /></div>
<div>
This post ended up larger than I was hoping for. Oh well.</div>Anonymoushttp://www.blogger.com/profile/07214376650672506391noreply@blogger.com0