ct-grid November 2008

ct-grid@nikhef.nl

10 participants
18 discussions

[Ct-grid] Ganga 5.1.1 deployed
by Hurng-Chun Lee 25 Nov '08

25 Nov '08

Hi all, Just to announce that the latest stable release of Ganga (v 5.1.1) has been deployed at NIKHEF. It's available on NIKHEF desktops (e.g. elel20). - The release note: http://ganga.web.cern.ch/ganga/release/5.1.1/release/ReleaseNotes-5.1.1 - configuration change at NIKHEF: * job submission to LCG is adopting the recently installed gLite WMS service at NIKHEF To start using it at NIKHEF: http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/Using_GANGA_with_A… comments and issue report: hn-atlas-dist-analysis-help(a)cern.ch Cheers, Hurng

1 0

[Ct-grid] Een idee voor ons nieuwste cluster?
by Tristan Suerink 24 Nov '08

24 Nov '08

http://tweakers.net/nieuws/56938/sgi-demonstreert-supercomputer-op-basis-va…

1 0

[Ct-grid] cannot start voms proxy on NIKHEF desktop
by Hurng-Chun Lee 24 Nov '08

24 Nov '08

hello, when doing "voms-proxy-init --vo atlas" on NIKHEF desktops (e.g. elel20), I got the following error: === [hclee@lxplus217:~/athena]$ cd .. GSS Major Status: Authentication Failed GSS Minor Status Error Chain: globus_gss_assist: Error during context initialization globus_gsi_callback_module: Could not verify credential globus_gsi_callback_module: Could not verify credential globus_gsi_callback_module: Invalid CRL: The available CRL has expired Trying next server for atlas. Creating temporary proxy ...................................... Done Contacting voms.cern.ch:15001 [/DC=ch/DC=cern/OU=computers/ CN=voms.cern.ch] "atlas" Failed Error: Could not establish authenticated connection with the server. GSS Major Status: Authentication Failed GSS Minor Status Error Chain: globus_gss_assist: Error during context initialization globus_gsi_callback_module: Could not verify credential globus_gsi_callback_module: Could not verify credential globus_gsi_callback_module: Invalid CRL: The available CRL has expired None of the contacted servers for atlas were capable of returning a valid AC for the user. === could you please have a look? Thanks! Cheers, Hurng

1 0

[Ct-grid] iSGTW - International Science Grid This Week
by Dennis van Dok 17 Nov '08

17 Nov '08

Hi *, for those of you who haven't heard of it yet, the iSGTW is a very nice weekly newsletter on grid computing. http://www.isgtw.org/ Cheers, Dennis -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] Reminder: Big Grid meeting today at 13.30
by Dennis van Dok 17 Nov '08

17 Nov '08

The next Big Grid meeting will be *today* (17 November) in H3.31 at 13.30. The agenda: - operations summary - middleware - applications - AOB Sorry for the late announcement. I should use an automatic reminder to remind me to send the announcement :-) Until then, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

2 1

[Ct-grid] Upgrade Ganga installation at NIKHEF to 5.1.0
by Hurng-Chun Lee 12 Nov '08

12 Nov '08

Hi all, Just to announce that I have upgraded the Ganga installation at NIKHEF to the latest version, 5.1.0. The official release note can be find here: http://ganga.web.cern.ch/ganga/release/5.1.0/release/ReleaseNotes-5.1.0 I also changed the site-level configuration to adopt the new "DQ2SandboxCache" for handling the oversized sandbox and to use gLite WMS services at CERN (so that user will benefit from the enhanced bulk submission feature). Sourcing "/project/atlas/nikhef/ganga/etc/setup.(c)sh", you always get the "current (i.e. the latest stable)" release. If you want to keep using older versions, you could look into the directory "/project/ atlas/nikhef/ganga/install" to set the $PATH accordingly. A Ganga + AMAAthena tutorial wiki can be found here: http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/Using_GANGA_with_A… Although you can still send me issues, you are also "officially" invited to send them to hn-atlas-dist-analysis-help(a)cern.ch as we are running analysis shifts for EU and US time zones. Thanks! Cheers, Hurng

1 0

[Ct-grid] Thank you!!
by Hurng-Chun Lee 08 Nov '08

08 Nov '08

Thank You Dear colleagues, Thank you all for the lovely flowers to Wen-Sheng. Best wishes, Wen-Sheng, Wan-Ching and Hurng-Chun

1 0

[Ct-grid] fork: Resource temporarily unavailable
by Hurng-Chun LEE 06 Nov '08

06 Nov '08

Hello, just recently, users on ribble get from time to time the error message like: "fork: Resource temporarily unavailable". I also get this error when ssh-ing to ribble. This is quite abnormal. Could you please have a look? Thanks! Cheers, Hurng

2 5

Re: [Ct-grid] fork: Resource temporarily unavailable
by Hurng-Chun LEE 06 Nov '08

06 Nov '08

Hi Jochem, On Nov 6, 2008, at 4:59 PM, Jochem Snuverink wrote: > Hi Hrung, > > On ribble there is a limit on the number of sessions per user and > therefore, I guess, also on the number of processes. That is likely > to be > the reason for the error message. is the limitation also on a group level? because I sometimes get this and sometimes not ... just by repeating same system command (without creating new sessions or processes ...). > Logging from ribble onto your own > machine will cure the problem. can you explain it more clearly? Thanks! Cheers, Hurng > > > Cheers, Jochem > > On Thu, 6 Nov 2008, Hurng-Chun LEE wrote: > >> Hello, >> >> just recently, users on ribble get from time to time the error >> message like: >> "fork: Resource temporarily unavailable". I also get this error >> when ssh-ing >> to ribble. >> >> This is quite abnormal. Could you please have a look? Thanks! >> >> Cheers, Hurng >>

1 0

[Ct-grid] Fwd: [Tier1-ams] data deletion - experience from ATLAS
by Jeff Templon 06 Nov '08

06 Nov '08

Hi, posted to the wrong list :-) Begin forwarded message: > From: Hurng-Chun Lee <Hurng-Chun.Lee(a)cern.ch> > Date: November 6, 2008 11:46:34 GMT+01:00 > To: <silvia(a)science.uva.nl> > Cc: tier1-ams(a)sara.nl > Subject: [Tier1-ams] data deletion - experience from ATLAS > Reply-To: tier1-ams(a)sara.nl > > Hi Silvia, > > The following points summarize what I used to do (and what ATLAS is > doing) for cleaning up the grid storage at SARA/NIKHEF. I don't know > if this will be useful for you so just for your information. > > === > firstly, ATLAS has it's own data management model based on the > concept of "dataset". A dataset is a bundle of the grid files with > same physics properties (e.g. files are generated by the same > production run or derived from the same physics stream). To maintain > the link between the ATLAS dataset and the grid files, ATLAS has its > own data management system called "DDM". DDM consists of a service, > a metadata catalogue and a set of client tools. Users use the client > tools to send requests to the service for certain data management > operations (metadata lookup, data migration, deletion, etc.); while > the service works together with the metadata catalogue to execute > the operations on behalf of the users. > > As "DDM" is the official data management tool of ATLAS, all the > users (both data production and data analysis users) are requested > to use it for all kind of data management activities (production, > replication, deletion, etc.). By this, all the ATLAS data on the > grid can be traced by the central operation team. Through the > metadata stored in the DDM catalogue, the ATLAS central operation > knows how many data has been produced around the world by ATLAS. > > secondly, ATLAS fully adopts the SRMv2 space token to manage the > storage areas on a site. Certain type of data is stored on a > particular storage area identified by a space token (by seeing the > token id, one has the idea about what kind of ATLAS data should be > stored there). The SRMv2 service also provides a "uniform" > management interface for heterogeneous storages at sites as it > supposes to be. Through the SRMv2 interface, the ATLAS central > operation can actually list what are sitting on a site and knows how > much space has been used by ATLAS. > > the first point shows the data from ATLAS point of view; while the > second point gives you the data physically stored at a site. By > comparing these two, one can make some consistency check and also > perform some deletion accordingly. Of course, there is another layer > for SRMv2 to local storage name space mapping to be checked; but for > ATLAS, this is a site issue (or let's say the storage technology > issue) should be done by the sites (or storage technicians). > > Unforunately, if some ATLAS user tried not to use "DDM" to do data > management operations, their production will not be considered as > the ATLAS data (as their data not registered in DDM catalogue). In > this case, they are considered as the dark data polluting the ATLAS > storage area. By policy, this kind of data will be removed without > any notification to the users (as it's also technically difficult to > identify the owner of the dark data). > > There was one time I tried to do some consistency check with NIKHEF > and SARA site admins to remove files not recognized by ATLAS (i.e. > files not presented in the ATLAS DDM). What we did was that I > firstly dumped a list of files (i.e. SURLs) that ATLAS thinks to be > at NIKHEF/SARA based on the DDM catalogue, secondly asked site > admins to dump a list of files (i.e. the name space) from the > corresponding storage area with local tools, and then compared two > lists to get the final deletion list (we also double checked the > list to ensure the important data are not deleted by accident). We > then cleaned up the LFC and the storage according to the deletion > list. We did it manually with site admin because by that time we > were cleaning old storage area without SRMv2 interface to query what > are located at site; and it was a massive deletion on site and > better to use local tools for better performance. > > I also have an impression that there must be some scripts already > done by ATLAS to automate the procedure as the consistency check is > run periodically by the central operation. If you have interests, I > can try to ask the experts on this. > === > > Hurng_______________________________________________ > tier1-ams mailing list > tier1-ams(a)horus.sara.nl > https://horus.sara.nl/mailman/listinfo/tier1-ams

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

ct-grid November 2008