Pith. sign in

REVIEW

The IceProd Framework: Distributed Data Processing for the IceCube Neutrino Observatory

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1311.5904 v3 pith:VSVYE2F3 submitted 2013-11-22 cs.DC

The IceProd Framework: Distributed Data Processing for the IceCube Neutrino Observatory

M. G. Aartsen , R. Abbasi , M. Ackermann , J. Adams , J. A. Aguilar , M. Ahlers , D. Altmann , C. Arguelles
show 279 more authors
J. Auffenberg X. Bai M. Baker S. W. Barwick V. Baum R. Bay J. J. Beatty J. Becker Tjus K.-H. Becker S. BenZvi P. Berghaus D. Berley E. Bernardini A. Bernhard D. Z. Besson G. Binder D. Bindig M. Bissok E. Blaufuss J. Blumenthal D. J. Boersma C. Bohm D. Bose S. B\"oser O. Botner L. Brayeur H.-P. Bretz A. M. Brown R. Bruijn J. Casey M. Casier D. Chirkin A. Christov B. Christy K. Clark L. Classen F. Clevermann S. Coenders S. Cohen D. F. Cowen A. H. Cruz Silva M. Danninger J. Daughhetee J. C. Davis M. Day C. De Clercq S. De Ridder P. Desiati K. D. de Vries M. de With T. DeYoung J. C. D\'iaz-V\'elez M. Dunkman R. Eagan B. Eberhardt B. Eichmann J. Eisch S. Euler P. A. Evenson O. Fadiran A. R. Fazely A. Fedynitch J. Feintzeig T. Feusels K. Filimonov C. Finley T. Fischer-Wasels S. Flis A. Franckowiak K. Frantzen T. Fuchs T. K. Gaisser J. Gallagher L. Gerhardt L. Gladstone T. Gl\"usenkamp A. Goldschmidt G. Golup J. G. Gonzalez J. A. Goodman D. G\'ora D. T. Grandmont D. Grant P. Gretskov J. C. Groh A. Gro{\ss} C. Ha A. Haj Ismail P. Hallen A. Hallgren F. Halzen K. Hanson D. Hebecker D. Heereman D. Heinen K. Helbing R. Hellauer S. Hickford G. C. Hill K. D. Hoffman R. Hoffmann A. Homeier K. Hoshina F. Huang W. Huelsnitz P. O. Hulth K. Hultqvist S. Hussain A. Ishihara E. Jacobi J. Jacobsen K. Jagielski G. S. Japaridze K. Jero O. Jlelati B. Kaminsky A. Kappes T. Karg A. Karle M. Kauer J. L. Kelley J. Kiryluk J. Kl\"as S. R. Klein J.-H. K\"ohne G. Kohnen H. Kolanoski L. K\"opke C. Kopper S. Kopper D. J. Koskinen M. Kowalski M. Krasberg A. Kriesten K. Krings G. Kroll J. Kunnen N. Kurahashi T. Kuwabara M. Labare H. Landsman M. J. Larson M. Lesiak-Bzdak M. Leuermann J. Leute J. L\"unemann O. Mac\'ias J. Madsen G. Maggi R. Maruyama K. Mase H. S. Matis F. McNally K. Meagher M. Merck G. Merino T. Meures S. Miarecki E. Middell N. Milke J. Miller L. Mohrmann T. Montaruli R. Morse R. Nahnhauer U. Naumann H. Niederhausen S. C. Nowicki D. R. Nygren A. Obertacke S. Odrowski A. Olivas A. Omairat A. O'Murchadha L. Paul J. A. Pepper C. P\'erez de los Heros C. Pfendner D. Pieloth E. Pinat J. Posselt P. B. Price G. T. Przybylski M. Quinnan L. R \"adel I. Rae M. Rameez K. Rawlins P. Redl R. Reimann E. Resconi W. Rhode M. Ribordy M. Richman B. Riedel J. P. Rodrigues C. Rott T. Ruhe B. Ruzybayev D. Ryckbosch S. M. Saba H.-G. Sander M. Santander S. Sarkar K. Schatto F. Scheriau T. Schmidt M. Schmitz S. Schoenen S. Sch\"oneberg A. Sch\"onwald A. Schukraft L. Schulte D. Schultz O. Schulz D. Seckel Y. Sestayo S. Seunarine R. Shanidze C. Sheremata M. W. E. Smith D. Soldin G. M. Spiczak C. Spiering M. Stamatikos T. Stanev N. A. Stanisha A. Stasik T. Stezelberger R. G. Stokstad A. St\"o{\ss}l E. A. Strahler R. Str\"om N. L. Strotjohann G. W. Sullivan H. Taavola I. Taboada A. Tamburro A. Tepe S. Ter-Antonyan G. Te\v{s}i\'c S. Tilav P. A. Toale M. N. Tobin S. Toscano M. Tselengidou E. Unger M. Usner S. Vallecorsa N. van Eijndhoven A. Van Overloop J. van Santen M. Vehring M. Voge M. Vraeghe C. Walck T. Waldenmaier M. Wallraff Ch. Weaver M. Wellons C. Wendt S. Westerhoff N. Whitehorn K. Wiebe C. H. Wiebusch D. R. Williams H. Wissing M. Wolf T. R. Wood K. Woschnagg D. L. Xu X. W. Xu J. P. Yanez G. Yodh S. Yoshida P. Zarzhitsky J. Ziemann S. Zierke M. Zoll
This is my paper
classification cs.DC
keywords icecubedataiceprodprocessingdetectordistributedframeworkmanagement
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. IceProd is a distributed management system based on Python, XML-RPC and GridFTP. It is driven by a central database in order to coordinate and admin- ister production of simulations and processing of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, Condor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.