ct-grid

ct-grid@nikhef.nl

243 discussions

[Ct-grid] iSGTW - International Science Grid This Week
by Dennis van Dok 17 Nov '08

17 Nov '08

Hi *, for those of you who haven't heard of it yet, the iSGTW is a very nice weekly newsletter on grid computing. http://www.isgtw.org/ Cheers, Dennis -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] Reminder: Big Grid meeting today at 13.30
by Dennis van Dok 17 Nov '08

17 Nov '08

The next Big Grid meeting will be *today* (17 November) in H3.31 at 13.30. The agenda: - operations summary - middleware - applications - AOB Sorry for the late announcement. I should use an automatic reminder to remind me to send the announcement :-) Until then, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

2 1

[Ct-grid] Upgrade Ganga installation at NIKHEF to 5.1.0
by Hurng-Chun Lee 12 Nov '08

12 Nov '08

Hi all, Just to announce that I have upgraded the Ganga installation at NIKHEF to the latest version, 5.1.0. The official release note can be find here: http://ganga.web.cern.ch/ganga/release/5.1.0/release/ReleaseNotes-5.1.0 I also changed the site-level configuration to adopt the new "DQ2SandboxCache" for handling the oversized sandbox and to use gLite WMS services at CERN (so that user will benefit from the enhanced bulk submission feature). Sourcing "/project/atlas/nikhef/ganga/etc/setup.(c)sh", you always get the "current (i.e. the latest stable)" release. If you want to keep using older versions, you could look into the directory "/project/ atlas/nikhef/ganga/install" to set the $PATH accordingly. A Ganga + AMAAthena tutorial wiki can be found here: http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/Using_GANGA_with_A… Although you can still send me issues, you are also "officially" invited to send them to hn-atlas-dist-analysis-help(a)cern.ch as we are running analysis shifts for EU and US time zones. Thanks! Cheers, Hurng

1 0

[Ct-grid] Thank you!!
by Hurng-Chun Lee 08 Nov '08

08 Nov '08

Thank You Dear colleagues, Thank you all for the lovely flowers to Wen-Sheng. Best wishes, Wen-Sheng, Wan-Ching and Hurng-Chun

1 0

[Ct-grid] fork: Resource temporarily unavailable
by Hurng-Chun LEE 06 Nov '08

06 Nov '08

Hello, just recently, users on ribble get from time to time the error message like: "fork: Resource temporarily unavailable". I also get this error when ssh-ing to ribble. This is quite abnormal. Could you please have a look? Thanks! Cheers, Hurng

2 5

Re: [Ct-grid] fork: Resource temporarily unavailable
by Hurng-Chun LEE 06 Nov '08

06 Nov '08

Hi Jochem, On Nov 6, 2008, at 4:59 PM, Jochem Snuverink wrote: > Hi Hrung, > > On ribble there is a limit on the number of sessions per user and > therefore, I guess, also on the number of processes. That is likely > to be > the reason for the error message. is the limitation also on a group level? because I sometimes get this and sometimes not ... just by repeating same system command (without creating new sessions or processes ...). > Logging from ribble onto your own > machine will cure the problem. can you explain it more clearly? Thanks! Cheers, Hurng > > > Cheers, Jochem > > On Thu, 6 Nov 2008, Hurng-Chun LEE wrote: > >> Hello, >> >> just recently, users on ribble get from time to time the error >> message like: >> "fork: Resource temporarily unavailable". I also get this error >> when ssh-ing >> to ribble. >> >> This is quite abnormal. Could you please have a look? Thanks! >> >> Cheers, Hurng >>

1 0

[Ct-grid] Fwd: [Tier1-ams] data deletion - experience from ATLAS
by Jeff Templon 06 Nov '08

06 Nov '08

Hi, posted to the wrong list :-) Begin forwarded message: > From: Hurng-Chun Lee <Hurng-Chun.Lee(a)cern.ch> > Date: November 6, 2008 11:46:34 GMT+01:00 > To: <silvia(a)science.uva.nl> > Cc: tier1-ams(a)sara.nl > Subject: [Tier1-ams] data deletion - experience from ATLAS > Reply-To: tier1-ams(a)sara.nl > > Hi Silvia, > > The following points summarize what I used to do (and what ATLAS is > doing) for cleaning up the grid storage at SARA/NIKHEF. I don't know > if this will be useful for you so just for your information. > > === > firstly, ATLAS has it's own data management model based on the > concept of "dataset". A dataset is a bundle of the grid files with > same physics properties (e.g. files are generated by the same > production run or derived from the same physics stream). To maintain > the link between the ATLAS dataset and the grid files, ATLAS has its > own data management system called "DDM". DDM consists of a service, > a metadata catalogue and a set of client tools. Users use the client > tools to send requests to the service for certain data management > operations (metadata lookup, data migration, deletion, etc.); while > the service works together with the metadata catalogue to execute > the operations on behalf of the users. > > As "DDM" is the official data management tool of ATLAS, all the > users (both data production and data analysis users) are requested > to use it for all kind of data management activities (production, > replication, deletion, etc.). By this, all the ATLAS data on the > grid can be traced by the central operation team. Through the > metadata stored in the DDM catalogue, the ATLAS central operation > knows how many data has been produced around the world by ATLAS. > > secondly, ATLAS fully adopts the SRMv2 space token to manage the > storage areas on a site. Certain type of data is stored on a > particular storage area identified by a space token (by seeing the > token id, one has the idea about what kind of ATLAS data should be > stored there). The SRMv2 service also provides a "uniform" > management interface for heterogeneous storages at sites as it > supposes to be. Through the SRMv2 interface, the ATLAS central > operation can actually list what are sitting on a site and knows how > much space has been used by ATLAS. > > the first point shows the data from ATLAS point of view; while the > second point gives you the data physically stored at a site. By > comparing these two, one can make some consistency check and also > perform some deletion accordingly. Of course, there is another layer > for SRMv2 to local storage name space mapping to be checked; but for > ATLAS, this is a site issue (or let's say the storage technology > issue) should be done by the sites (or storage technicians). > > Unforunately, if some ATLAS user tried not to use "DDM" to do data > management operations, their production will not be considered as > the ATLAS data (as their data not registered in DDM catalogue). In > this case, they are considered as the dark data polluting the ATLAS > storage area. By policy, this kind of data will be removed without > any notification to the users (as it's also technically difficult to > identify the owner of the dark data). > > There was one time I tried to do some consistency check with NIKHEF > and SARA site admins to remove files not recognized by ATLAS (i.e. > files not presented in the ATLAS DDM). What we did was that I > firstly dumped a list of files (i.e. SURLs) that ATLAS thinks to be > at NIKHEF/SARA based on the DDM catalogue, secondly asked site > admins to dump a list of files (i.e. the name space) from the > corresponding storage area with local tools, and then compared two > lists to get the final deletion list (we also double checked the > list to ensure the important data are not deleted by accident). We > then cleaned up the LFC and the storage according to the deletion > list. We did it manually with site admin because by that time we > were cleaning old storage area without SRMv2 interface to query what > are located at site; and it was a massive deletion on site and > better to use local tools for better performance. > > I also have an impression that there must be some scripts already > done by ATLAS to automate the procedure as the consistency check is > run periodically by the central operation. If you have interests, I > can try to ask the experts on this. > === > > Hurng_______________________________________________ > tier1-ams mailing list > tier1-ams(a)horus.sara.nl > https://horus.sara.nl/mailman/listinfo/tier1-ams

1 0

[Ct-grid] Fwd: Doodle: "big grid weekly ops meeting"
by Jeff Templon 05 Nov '08

05 Nov '08

Hi, here is the poll. it is listed for the week of 17 november, don't worry about that, it is supposed to be every week so just consider what regular meetings you have to avoid. Please respond *promptly*, ie do not wait one week before filling in the poll, especially if you're the one who didn't like the original time :-) JT Begin forwarded message: > From: Doodle Mailer <mailer(a)doodle.com> > Date: November 4, 2008 13:47:17 GMT+01:00 > To: templon(a)nikhef.nl > Subject: Doodle: "big grid weekly ops meeting" > Reply-To: templon(a)nikhef.nl > > You have initiated a poll for "big grid weekly ops meeting" at > Doodle. The link to your poll is: > > http://doodle.com/s297b2dp8xqiru9i > > Please send this link to all potential participants, and they will > be able to cast their votes. Do not forget to make your own > selection, too. > > If you did not initiate this poll, somebody must have used your e- > mail address by accident. Please ignore this mail.

1 1

[Ct-grid] I am back with a new family member
by Hurng-Chun Lee 05 Nov '08

05 Nov '08

Hello, I am just back from the vacation with a most important production, Wen- Sheng Lee, the first baby in my family. He was born on 25th October at 4:30 am, 3240 gram. (http://picasaweb.google.com/HurngChunLee/WenShengLee# ) I have some traditional Taiwanese candy to celebrate it (they were prepared by my mother). If you like, you could pass by my office to get it ;-) Cheers, Hurng

2 1

[Ct-grid] Announcement: LOFAR archive technical workshop, Amsterdam, 27 Nov 13.30-17.00
by Dennis van Dok 04 Nov '08

04 Nov '08

(re-sent because mailman is acting up, sorry if this is a dup) Hi all, on November 27th a technical workshop will be held in Amsterdam to discuss storage solutions for the LOFAR long term archive. This meeting is organised by the LOFAR archive working group, which is a short-lived task force to draft a blue print for the archive's design before the end of 2008. In an earlier workshop with the LOFAR key science projects, we've discussed the requirements from the user's point of view. The focus of the upcoming meeting will be to reflect our findings with the mass storage experts. Could you please reply directly to me if you intend to come (if you've not already done so)? Thanks, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

1 0

← Newer
1
...
17
18
19
20
21
22
23
24
25
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

ct-grid