ct-grid August 2009

ct-grid@nikhef.nl

9 participants
10 discussions

[Ct-grid] Data centers too cool?
by Dennis van Dok 31 Aug '09

31 Aug '09

Interesting article; I know we're being conscious about energy/costs/etc. but the new Nikhef PDP data center is pretty cold, at least the inlet is way below 25degC. Intel wants you to know that data centers are wasting energy - and money - by over-cooling their servers, burdened by warranties that may prevent them from aggressively raising their temperature. http://www.theregister.co.uk/2009/08/31/data_centers_run_too_cool/ --Dennis

1 0

[Ct-grid] Next BiG Grid Meeting: Monday August 31 at 13.30h in room N3.28
by Dennis van Dok 28 Aug '09

28 Aug '09

The Next BiG Grid meeting will be held on Monday, August 31, 2009 from 13.30-14.30h in room *N3.28*. (Please take note of the different location.) This week's special: a presentation by Willem van Engen, titled: "Introducing jGridstart" Get started with the grid easily. jGridstart helps end-users to request, obtain, install and renew grid certificates in a friendly way. http://www.nikhef.nl/pub/projects/grid/gridwiki/index.php/JGridstart Agenda: - presentation by Willem - open action items * Jan Just: Look at CIC portal concerning downtime * Jan Just: Look at checksumming in GFAL. - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutes of the previous meeting(s) are located here: https://www.nikhef.nl/pub/departments/ct/grid/2009/ Minutes to be taken by Tristan. Enjoy the weekend, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 22 28 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] action point: Look at CIC portal concerning downtime notifications.
by Jan Just Keijser 28 Aug '09

28 Aug '09

hi all, the other action point from last monday's grid-overleg: why are downtime notifications sent only a day before they actually occur? The short answer is: this is exactly as specified. If you read https://edms.cern.ch/file/829986/0.1/EGEE-downtime-notification-procedure.p… which in turn refers to https://edms.cern.ch/file/829986/0.1/EGEE-intervention-procedures.pdf then most downtime notifications fall into category C: − Send broadcast 1 day in advance − Broadcast targets: all ROCs, affected VOs, all production sites in the related ROC so the actual broadcast happens only 1 day in advance !! of course , this makes the downtime notification procedure kinda useless for us. As far as I see there are 2 options: 1) create an email list infrastructure-announce(a)biggrid.nl to which all BigGrid downtime notifications must be sent (on penalty of death-by-tickling or something like that) 2) the GOCDB, where all site downtimes are entered, luckily has a programming interface where you can retrieve all downtimes for a particular site after a particular date: we could thus create a cron job that periodically checks the GOCDB and broadcasts any messages , as appropriate. For example, if I do curl --cert $X509_USER_PROXY --key $X509_USER_PROXY --CApath $X509_CERT_DIR -k \ "https://goc.gridops.org/gocdbpi/public/?method=get_downtime&topentity=SARA-…" I get <?xml version="1.0"?> <ROOT> <DOWNTIME ID="45205458" CLASSIFICATION="SCHEDULED"> <HOSTNAME>srm.grid.sara.nl</HOSTNAME> <SEVERITY>OUTAGE</SEVERITY> <DESCRIPTION> The tape backend will be down for the whole day due to maintenance. srm.grid.sara.nl will be up but data on tape cannot be accessed. </DESCRIPTION> <START_DATE>1251705600</START_DATE> <END_DATE>1251748800</END_DATE> <FORMATED_START_DATE>2009-08-31 08:00</FORMATED_START_DATE> <FORMATED_END_DATE>2009-08-31 20:00</FORMATED_END_DATE> </DOWNTIME> <DOWNTIME ID="43305437" CLASSIFICATION="SCHEDULED"> <HOSTNAME>ce.gina.sara.nl</HOSTNAME> <SEVERITY>OUTAGE</SEVERITY> <DESCRIPTION> maintenance to broken switch hardware </DESCRIPTION> <START_DATE>1251190800</START_DATE> <END_DATE>1251219600</END_DATE> <FORMATED_START_DATE>2009-08-25 09:00</FORMATED_START_DATE> <FORMATED_END_DATE>2009-08-25 17:00</FORMATED_END_DATE> </DOWNTIME> </ROOT> so this API _does_ work... now we need a little XML scripting magic to turn it into a broadcast email - any takers? The upside of this approach is that we have a single entry point for downtimes , i.e. less chance of errors. share and enjoy, JJK

4 3

[Ct-grid] BiG Grid meetings in September
by Dennis van Dok 27 Aug '09

27 Aug '09

As you know, the twice-a-month grid meetings are scheduled regularly on the 1st and 3rd Monday of each month. But the third week of September is special: due to the EGEE'09 conference in Barcelona there will be no grid meeting. Therefore I propose to move the meetings forward one week: so Monday 31 August (coming Monday!) and Monday 14 September I will send a reminder on Friday afternoon. Best, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 22 28 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] Tristan Glatard (vlemed) praatje om 10.00 vandaag (Nikehf H3.31)
by David Groep 26 Aug '09

26 Aug '09

Komt allen (zeker gezien de titel :-) Dear colleague, Tristan Glatard, a CNRS researcher at the Creatis-LRMN lab and a former member of VL-e, will present a talk at the Science Park next week. Looking forward to meeting you there, -- Silvia D. Olabarriaga Academic Medical Center +31 20 566 4660 (Tue-Fri) Informatics Institute +31 20 525 7549 (Mon) University of Amsterdam http://www.science.uva.nl/~silvia/contact.html =============================== My after-Dutch life: How much I miss grid.support ;-) This talk will present our recent design, implementation, application porting and experimental efforts conducted at Creatis (Lyon, France) related to VL-e medical software. We will first present an integration of DIANE pilot jobs to improve quality of service in the biomed VO. Then, a service performing site monitoring and pre-selection will be described and experimental results will show how it complements pilot jobs. Besides, an interface of VL-e medical software with the ARC middleware will be presented, allowing application workflows to run both on EGEE/gLite and NorduGrid? /ARC resources. Finally, three application use-cases (namely FIELD ultrasonic simulation, GATE radiotherapy simulation and CAVIAR cardiovascular analysis) will be presented, illustrating current achievements, remaining technical issues and on-going optimizations. When & Where: * 26 August, 2009 * 10-11h * NIKHEF Room H3.31 More info on http://amc-app1.amc.sara.nl/twiki/bin/view/EBioScience/News#GlatardScienceP… This event is jointly organized by VL-e, Big Grid and the e-Bioscience group of the Bioinformatics Laboratory of the AMC. -- David Groep ** Nikhef, Dutch National Institute for Sub-atomic Physics,PDP/Grid group ** ** Room: H1.50 Phone: +31 20 5922179, PObox 41882, NL-1009DB Amsterdam NL **

1 0

[Ct-grid] action point: Look at checksumming in GFAL
by Jan Just Keijser 22 Aug '09

22 Aug '09

hi all, here's a quick rundown of checksumming in GFAL and the various storage systems used throughout biggrid: dCache ====== - source code not available - support (HEP-preferred?) ADLER32 checksums only - checksum is calculated when the file is uploaded and stored in a database; this means that it is not possible to verify whether the file on disk/tape actually matches the checksum. A dCache operator can trigger a command to re-calculate the checksum DPM ==== - supports ADLER32, MD5 and CRC32 checksums - checksum is calculated if not present in the DPM namespace and then stored there; this means that the first time you request a checksum it can take a while (and can put quite some stress on the DPM worker nodes) - uses a plain gridftp server with a DPM plugin to calculate the ADLER32 and CRC32 checksums. - I browsed through the sources to check/find out some of this StoRM ===== - does not seem to support checksums in a consistent manner at all - uses a plain gridftp server which only supports MD5 checksums yet lcg-cr .... --checksum --checksum-type md5 always fails - it is possible to calculate the checksum of a file on the storm gridftp server but it is not stored anywhere for fast access: this makes it ideal for a Denial-of-Server-Storage attack :) - is actually only SRMv2.1 compliant - also does not support ACLs/permissions - no new version has been released since 2007 - I browsed through the sources to check/find out some of this Hopefully I've missed a few things on the storm front, otherwise things don't look too good ... share and enjoy, JJK

4 6

[Ct-grid] Fwd: CERN Computing Colloquium announcement: August 21st, 2009 (fwd)
by Hurng-Chun Lee 20 Aug '09

20 Aug '09

F.Y.I. keywords: Hadoop, HEP, data analysis, Cloud, ... Begin forwarded message: > From: Alexei Klimentov <alexei(a)mail.cern.ch> > Date: August 19, 2009 6:56:57 PM GMT+02:00 > To: ADC Operations <atlas-project-adc-operations(a)cern.ch> > Subject: CERN Computing Colloquium announcement: August 21st, 2009 > (fwd) > > > FYI > >> >>> From: David Myers <David.Myers(a)cern.ch> >>> Date: 12 August 2009 09:06:03 GMT+02:00 >>> To: "it-dep-full (All the members in IT Department in its extended >>> form)" <it-dep-full(a)cern.ch> >>> Subject: CERN Computing Colloquium announcement: August 21st, 2009 >>> >>> _____________________________________________ >>> Dear All, >>> >>> The next Computing Colloquium will take place: >>> >>> On: Friday , August 21st >>> At: 14:00 >>> In: Main Auditorium >>> >>> Title: Analyzing petabytes of data with Hadoop >>> >>> Speaker: Mr. Jeff Hammerbacher, Chief Scientist, Cloudera >>> >>> Abstract: The open source Apache Hadoop project provides a powerful >>> suite of tools for storing and analyzing petabytes of data using >>> commodity hardware. After several years of production use inside of >>> web companies like Yahoo! and Facebook and nearly a year of >>> commercial support and development by Cloudera, the technology is >>> spreading rapidly through other disciplines, from financial >>> services and government to life sciences and high energy physics. >>> >>> The talk will motivate the design of Hadoop and discuss some key >>> implementation details in depth. It will also cover the major >>> subprojects in the Hadoop ecosystem, go over some example >>> applications, highlight best practices for deploying Hadoop in your >>> environment, discuss plans for the future of the technology, and >>> provide pointers to the many resources available for learning more. >>> >>> In addition to providing more information about the Hadoop >>> platform, a major goal of this talk is to begin a dialogue with the >>> ATLAS research team on how the tools commonly used in their >>> environment compare to Hadoop, and how Hadoop could improve better >>> to serve the high energy physics community. >>> >>> Short Biography: Jeff Hammerbacher is Vice President of Products >>> and Chief Scientist at Cloudera. Jeff was an Entrepreneur in >>> Residence at Accel Partners immediately prior to founding Cloudera. >>> Before Accel, he conceived, built, and led the Data team at >>> Facebook. The Data team was responsible for driving many of the >>> applications of statistics and machine learning at Facebook, as >>> well as building out the infrastructure to support these tasks for >>> massive data sets. The team produced two open source projects: >>> Hive, a system for offline analysis built above Hadoop, and >>> Cassandra, a structured storage system on a P2P network. Before >>> joining Facebook, Jeff was a quantitative analyst on Wall Street. >>> Jeff earned his Bachelor's Degree in Mathematics from Harvard >>> University and recently served as contributing editor to the book >>> "Beautiful Data", published by O'Reilly in July 2009. >>> >>> The InDiCo link is here: http://indico.cern.ch/conferenceDisplay.py?confId=59791 >>> >>> With regards, >>> >>> David Myers. >>> >>> -------------- European Organization for Nuclear Research >>> -------------- >>> Dr. D.R. Myers, Tel: +41 22 767 4646 >>> Head, Computing Security, Mobile: +41 76 487 3994 >>> CERN, IT Division, >>> CH-1211 Geneva 23, Switzerland. EMail: David.Myers(a)cern.ch >>> ------------------------------------------------------------------------ >>> >>> >>>

1 0

[Ct-grid] Next BiG Grid Meeting: Monday 17 August, 13:30h in H3.31
by Dennis van Dok 14 Aug '09

14 Aug '09

The Next BiG Grid meeting will be held on Monday 17 August, from 13.30-14.30h in room H3.31. Agenda: - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutes of the previous meeting(s) are located here: https://www.nikhef.nl/pub/departments/ct/grid/2009/ I wish you a joyous weekend, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 22 28 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] koffie en taart in H122 om 10:00
by Kees Huyser 13 Aug '09

13 Aug '09

Omdat ik tegenwoordig steeds jonger wordt (ik komt nl. elk jaar verder van mijn pensioen af!) vier ik dit met koffie en taart in H122 om 10:00. Je bent van harte welkom! --kees

1 0

[Ct-grid] REMINDER: next BiG Grid Meeting: Monday 3 August 2009 13.30h in room H3.31
by Dennis van Dok 01 Aug '09

01 Aug '09

The Next BiG Grid meeting will be held on Monday 3 August 2009, from 13.30-14.30h in room H3.31. Agenda: - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutes of the previous meeting(s) can be found at: https://www.nikhef.nl/pub/departments/ct/grid/2009/ Enjoy the weekend, Dennis van Dok

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

ct-grid August 2009