ct-grid

ct-grid@nikhef.nl

243 discussions

[Ct-grid] ggus
by Jeff Templon 27 Oct '08

27 Oct '08

Hi * i think i promised you guys this information, about the "revolution" in GGUS. From Maria Dimou: ================= A new revolutionary GGUS development was recently decided at the User Support Advisory Group meeting. Normal GGUS tickets will be allowed to reach the Site's helpdesk address when a GOCDB-GGUS interface, based on Web Services, will be ready. A lot of technical (interface with GOCdb, interface with local ticketing systems) and procedural issues (what if the sites are not in GOCdb or have multiple email contact addresses per service) have to be clarified in this process. =============== I think this means that tickets (in many cases) get to skip the whole TPM thing. JT

2 1

[Ct-grid] BiG Grid meetings
by Dennis van Dok 27 Oct '08

27 Oct '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi *, Today's meeting was mostly a dud for lack of operations people and another failing video conference. There was some concern that the change of scope of this meeting will see a diminished attendance of the operations people in the future; I sincerely hope that they will keep coming even though they have their own meeting now. Let's try again next week; this should bring us 'out of phase' with the monthly departmental meeting at Sara. Proposed dates: 3 Nov 17 Nov 1 Dec 15 Dec Mondays at 13.30h. Please let me know if there are structural problems with these dates. Best, Dennis - -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJBchsIITq5lEwLHcRAmsuAJ9olrCpYbM5leO/7xTAsN7dRsAbKgCdGp1m kVlcytztkNNS+u8i8NwlHO0= =TzOx -----END PGP SIGNATURE-----

1 0

[Ct-grid] nikhef website
by Jan Just Keijser 27 Oct '08

27 Oct '08

Hoi Kees, de URL http://www.nikhef.nl/grid/gridwiki/ werkt niet meer sinds de overgang naar de nieuwe site: Sorry! The wiki is experiencing some technical difficulties, and cannot contact the database server. Access denied for user 'atlaswiki'@'ferro.nikhef.nl' (using password: YES) JJK

2 1

[Ct-grid] Next BiG Grid meeting
by Dennis van Dok 27 Oct '08

27 Oct '08

The next grid meeting will be on Monday 27 October, at *13.00* in room *N3.28*. Be aware that both time and place are different than usual. Video conference will be available. The preliminary agenda: - operations summary - middleware - applications Minutes will be taken by Gerben. Hope to see you there, Dennis -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

2 1

[Ct-grid] egee/wlcg intervention procedures
by Jeff Templon 23 Oct '08

23 Oct '08

Hi *, Ik had beloofd om dit rond te sturen: https://edms.cern.ch/cedar/plsql/doc.info?cookie=7918831&document_id=829986… Jullie hadden gelijk: in het EGEE geval, voor minder dan 4 uur down, is een broadcast, 1 dag van te voren, voldoende. JT

1 0

[Ct-grid] Fwd: Re: next grid meeting
by Oscar Koeroo 23 Oct '08

23 Oct '08

Subject: Re: [Ct-grid] next grid meeting From: Tristan Glatard <tristan.glatard(a)wanadoo.fr> Date: Mon, 20 Oct 2008 10:57:15 +0200 To: "ct-grid(a)nikhef.nl" <ct-grid(a)nikhef.nl> CC: "Silvia D. Olabarriaga" <silvia(a)science.uva.nl> Hoi, vlemed has nothing special to report to the meeting, except a good 100% success ratio: we've submitted 1292 jobs last week and they were all successful. Cheers, Tristan Ronald Starink wrote: > The next grid meeting will be on Monday October 20 at 11:00 in room > H3.31. The video-conf setup has been tested and worked well, so we'll > give it a try. > > Agenda: > - Gridview & Infrastructure > - Middleware > - Applications > - Any Other Business > > The minutes of previous meetings are available at: > http://www.nikhef.nl/pub/departments/ct/grid/ > > Minutes will be taken by: Sven > > Enjoy the weekend, > Ronald > > > > > _______________________________________________ > Ct-grid mailing list > Ct-grid(a)nikhef.nl > https://mailman.nikhef.nl/cgi-bin/listinfo/ct-grid > > -- ------------------------------------------------------------- Tristan Glatard Institute of Informatics / Amsterdam Medical Center University of Amsterdam http://staff.science.uva.nl/~glatard

1 0

[Ct-grid] Fwd: [Sp-leaders] registratie VL-e workshop. Herinnering!!!
by Jeff Templon 22 Oct '08

22 Oct '08

Hi wie gaat er naar de VL-e workshop? registreren, mijn koekje mag je niet opeten. Begin forwarded message: > From: Jacqueline van der Velde <J.H.vanderVelde(a)uva.nl> > Date: October 22, 2008 13:08:06 GMT+02:00 > To: sp-leaders(a)lists.vl-e.nl, Ammar Benabdelkader <ammar(a)science.uva.nl > >, "Guido van 't Noordende" <noordend(a)science.uva.nl>, "M. Scott > Marshall" <marshall(a)science.uva.nl> > Subject: [Sp-leaders] registratie VL-e workshop. Herinnering!!! > > LS, > Tot nu hebben zich de volgende mensen officieel bij mijn aangemeld. > Kunt u aub de mensen uit uw groep er aan herinneren zich te > registreren. A.s. vrijdag moet ik nl. de catering regelen. > Mvgr > Jacqueline > > > > Matthijs Koot - goup Cees > Jeff Templon > Henk Sips - TUDelft > Keith Cover - VUMC > Ronald Schut - PCC > Maik Nijhuis - VU > Ana-Maria Oprescu - VU > Kees Verstoep - VU > Ceriel Jacobs - VU > Nick Palmer - VU > Lodewijk Bogaards - DANS > Roelof Kemp - VU > Dick Epema - TUDelft > Victor Guevara - UvA > Dirk Out - Unilever > Arie Kaizer - Philips > Ronald van Driel - Philips > Willem de Bruijn - VU > Thilo Kielmann - VU > Alexandru Iosup - TUDelft > Arthur van Schijndel - HP (via Ammar) > Dick Kamphuis - HP (via Ammar) > Kees van Reeuwijk - VU > Bart Grundeken - TU Delft > _______________________________________________ > Sp-leaders mailing list > Sp-leaders(a)lists.vl-e.nl > https://lists.vl-e.nl/mailman/listinfo/sp-leaders

2 2

[Ct-grid] Bugje in kernel kan je netwerkkaart verklooien
by Oscar Koeroo 20 Oct '08

20 Oct '08

Getroffen NIC type e1000e. Vooral in Ubuntu, OpenSUSE 11 en andere distro's met 2.6.27 inclusief rc7. http://tweakers.net/nieuws/55854/grote-linux-distros-getroffen-door-netwerk… Persoonlijk ken ik het e1000 labeltje vooral van die Dell 1950's of zit ik er naast? Deze kernel zou ik overslaan. groet, Oscar

3 3

[Ct-grid] new poll for dates of the grid meeting
by Dennis van Dok 20 Oct '08

20 Oct '08

Hi, some of you have already voiced objections against Tuesday mornings... There is now a Doodle poll, where everyone can enter his/her preferences. I've listed Monday through Friday, and for each day there is a 'before lunch' and 'after lunch' option. Please indicate the when you are 'mostly' available to attend this meeting. http://doodle.ch/participation.html?pollId=ii6z7kyxnizgym95 By end of this week it should be clear which day/time to pick as our new default. Thanks, Dennis -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 50 12 :: http://www.nikhef.nl/~dennisvd/

1 0

Re: [Ct-grid] FW: [Condor-team] Hadoop and Condor
by Oscar Koeroo 20 Oct '08

20 Oct '08

Hi Brian, The performance and scalability of HDFS is very interesting. A few questions: - How do you mount HDFS on a system? Is FUSE the only option for a regular mount? As I understand the information on the Hadoop website it would require application to use the Java Virtual FS bindings to Hadoop FS or redirect trough FUSE. - If FUSE is the option, what would the performance penalty be? - For Grid-enabling HDFS do you refer to the Java version of the GridFTPd or the C implementation? thanks, Oscar Brian Bockelman wrote: > Hey Jeff, > > I'd be happy to come and talk about Hadoop and quantify my comments. It > really is an amazing, well-engineered file system. We currently have a > 30TB install (completely from scratch space in worker node disks). Some > of the highlights: > - Resource consumption: If I push it as hard as I can (several hundred > CMSSW processes reading the same file via POSIX I/O), it maybe uses a > 1/3 of a CPU. Can't make its memory consumption show up in graphs. > - Performance: Doing the same test with *one file* and 2 replicas, I can > hit 80Gbps. Yahoo claims that their biggest clusters get up to > 2.2Tbps. I haven't been able to make the namenode break a sweat > - Manageability: Straightforward. The namenode is journaled, and can > checkpoint. There's only two types of nodes (namenode and datanode); > both are straightforward to manage. Out of the box, there are no knobs > to tweak, no optimizations to perform to get decent performance. Just > works. If you lose a data node, it starts re-replicating files within > about 10 seconds. [As a side note, data recovery is actually pretty > straightforward. We lost the disk which holds the namenode's info this > weekend, and restoring from an automatically-created checkpoint was easy.] > > If you want to grid-enable it, you reuse the Globus GridFTP server (plus > bindings) and the BeStMan SRM server. I've clocked it at 8.2Gbps for > WAN traffic. > > For us, the best potential upside is the manageability, although the > fact that performance-wise, it does things dCache can't do. If it can > prove to require far less admin time, we are hoping to expand use beyond > the current test bed. > > Brian > > > On Oct 18, 2008, at 1:50 AM, Jeff Templon wrote: > >> Yo, >> >> I'm talking to Brian Bockleman about other things, maybe if I can >> convince him to come, he can tell us about his experiences with HDFS. >> >> Brian, now you have another reason to come ... give us a short >> technical seminar about HDFS? It could just be you with a piece of >> chalk and a blackboard, as informal as you like? >> >> JT >> >> Oscar Koeroo wrote: >>> Hi Sander, >>> I'm very interested in the details, opportunities and experiences with >>> Hadoop, but I don't have a clue about any of these things. >>> Hadoop: http://hadoop.apache.org/core/ >>> Hadoop FAQ >>> 1. What is Hadoop? >>> [WWW] Hadoop is a distributed computing platform written in Java. It >>> incorporates features similar to those of the [WWW] Google File System >>> and of [WWW] MapReduce. For some details, see HadoopMapReduce. >>> Oscar >>> Sander Klous wrote: >>>> Hi >>>> What is Hadoop exactly? Reading through the specs it doesn't look >>>> like a >>>> competitor for dCache. So, can anybody comment on the remark below: >>>> "HDFS is >>>> much better than dCache"? >>>> Thanks, >>>> Sander >>>> >>>>> Plan: >>>>> >>>>> 1) Gain experience w/ the Hadoop File System via a large >>>>> installation on the cluster in B240. >>>>> >>>>> 2) Assuming it really works according to our expectations, >>>>> investigate how to best "pseudo-daemoncore-ify" the HDFS service >>>>> so that runs under the condor_master, responds to Condor >>>>> administrative commands such as on, off, restart, reconfig, etc, >>>>> and hopefully has similar behavior for a debug log and config >>>>> settings. Note: some challenges here include HDFS service is >>>>> implemented in Java, perhaps relies on ssh for network >>>>> authentication, etc. >>>>> >>>>> 3) Enable the Condor file transfer service to stage files in/out >>>>> of HDFS, in addition to just the shadow. >>>> That's really interesting. >>>> >>>> As one datapoint: I was talking to Brian Bockelman about HDFS. He >>>> has been deployed HDFS at his OSG site in Nebraska, and thinks that >>>> it is great, much better than dCache. >>>> >>>> An interesting question, once we get far enough along with this >>>> work: can the interface to HDFS be made pluggable? That is, can >>>> Condor call out to a file transfer service? I don't know if it >>>> makes sense, but we might be able to plug in other services one day. >>>> >>>> >>>> _______________________________________________ >>>> Ct-grid mailing list >>>> Ct-grid(a)nikhef.nl >>>> https://mailman.nikhef.nl/cgi-bin/listinfo/ct-grid >>> ------------------------------------------------------------------------ >>> _______________________________________________ >>> Ct-grid mailing list >>> Ct-grid(a)nikhef.nl >>> https://mailman.nikhef.nl/cgi-bin/listinfo/ct-grid

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

ct-grid