ct-grid

ct-grid@nikhef.nl

243 discussions

[Ct-grid] BiG Grid meetings in September
by Dennis van Dok 27 Aug '09

27 Aug '09

As you know, the twice-a-month grid meetings are scheduled regularly on the 1st and 3rd Monday of each month. But the third week of September is special: due to the EGEE'09 conference in Barcelona there will be no grid meeting. Therefore I propose to move the meetings forward one week: so Monday 31 August (coming Monday!) and Monday 14 September I will send a reminder on Friday afternoon. Best, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 22 28 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] Tristan Glatard (vlemed) praatje om 10.00 vandaag (Nikehf H3.31)
by David Groep 26 Aug '09

26 Aug '09

Komt allen (zeker gezien de titel :-) Dear colleague, Tristan Glatard, a CNRS researcher at the Creatis-LRMN lab and a former member of VL-e, will present a talk at the Science Park next week. Looking forward to meeting you there, -- Silvia D. Olabarriaga Academic Medical Center +31 20 566 4660 (Tue-Fri) Informatics Institute +31 20 525 7549 (Mon) University of Amsterdam http://www.science.uva.nl/~silvia/contact.html =============================== My after-Dutch life: How much I miss grid.support ;-) This talk will present our recent design, implementation, application porting and experimental efforts conducted at Creatis (Lyon, France) related to VL-e medical software. We will first present an integration of DIANE pilot jobs to improve quality of service in the biomed VO. Then, a service performing site monitoring and pre-selection will be described and experimental results will show how it complements pilot jobs. Besides, an interface of VL-e medical software with the ARC middleware will be presented, allowing application workflows to run both on EGEE/gLite and NorduGrid? /ARC resources. Finally, three application use-cases (namely FIELD ultrasonic simulation, GATE radiotherapy simulation and CAVIAR cardiovascular analysis) will be presented, illustrating current achievements, remaining technical issues and on-going optimizations. When & Where: * 26 August, 2009 * 10-11h * NIKHEF Room H3.31 More info on http://amc-app1.amc.sara.nl/twiki/bin/view/EBioScience/News#GlatardScienceP… This event is jointly organized by VL-e, Big Grid and the e-Bioscience group of the Bioinformatics Laboratory of the AMC. -- David Groep ** Nikhef, Dutch National Institute for Sub-atomic Physics,PDP/Grid group ** ** Room: H1.50 Phone: +31 20 5922179, PObox 41882, NL-1009DB Amsterdam NL **

1 0

[Ct-grid] action point: Look at checksumming in GFAL
by Jan Just Keijser 22 Aug '09

22 Aug '09

hi all, here's a quick rundown of checksumming in GFAL and the various storage systems used throughout biggrid: dCache ====== - source code not available - support (HEP-preferred?) ADLER32 checksums only - checksum is calculated when the file is uploaded and stored in a database; this means that it is not possible to verify whether the file on disk/tape actually matches the checksum. A dCache operator can trigger a command to re-calculate the checksum DPM ==== - supports ADLER32, MD5 and CRC32 checksums - checksum is calculated if not present in the DPM namespace and then stored there; this means that the first time you request a checksum it can take a while (and can put quite some stress on the DPM worker nodes) - uses a plain gridftp server with a DPM plugin to calculate the ADLER32 and CRC32 checksums. - I browsed through the sources to check/find out some of this StoRM ===== - does not seem to support checksums in a consistent manner at all - uses a plain gridftp server which only supports MD5 checksums yet lcg-cr .... --checksum --checksum-type md5 always fails - it is possible to calculate the checksum of a file on the storm gridftp server but it is not stored anywhere for fast access: this makes it ideal for a Denial-of-Server-Storage attack :) - is actually only SRMv2.1 compliant - also does not support ACLs/permissions - no new version has been released since 2007 - I browsed through the sources to check/find out some of this Hopefully I've missed a few things on the storm front, otherwise things don't look too good ... share and enjoy, JJK

4 6

[Ct-grid] Fwd: CERN Computing Colloquium announcement: August 21st, 2009 (fwd)
by Hurng-Chun Lee 20 Aug '09

20 Aug '09

F.Y.I. keywords: Hadoop, HEP, data analysis, Cloud, ... Begin forwarded message: > From: Alexei Klimentov <alexei(a)mail.cern.ch> > Date: August 19, 2009 6:56:57 PM GMT+02:00 > To: ADC Operations <atlas-project-adc-operations(a)cern.ch> > Subject: CERN Computing Colloquium announcement: August 21st, 2009 > (fwd) > > > FYI > >> >>> From: David Myers <David.Myers(a)cern.ch> >>> Date: 12 August 2009 09:06:03 GMT+02:00 >>> To: "it-dep-full (All the members in IT Department in its extended >>> form)" <it-dep-full(a)cern.ch> >>> Subject: CERN Computing Colloquium announcement: August 21st, 2009 >>> >>> _____________________________________________ >>> Dear All, >>> >>> The next Computing Colloquium will take place: >>> >>> On: Friday , August 21st >>> At: 14:00 >>> In: Main Auditorium >>> >>> Title: Analyzing petabytes of data with Hadoop >>> >>> Speaker: Mr. Jeff Hammerbacher, Chief Scientist, Cloudera >>> >>> Abstract: The open source Apache Hadoop project provides a powerful >>> suite of tools for storing and analyzing petabytes of data using >>> commodity hardware. After several years of production use inside of >>> web companies like Yahoo! and Facebook and nearly a year of >>> commercial support and development by Cloudera, the technology is >>> spreading rapidly through other disciplines, from financial >>> services and government to life sciences and high energy physics. >>> >>> The talk will motivate the design of Hadoop and discuss some key >>> implementation details in depth. It will also cover the major >>> subprojects in the Hadoop ecosystem, go over some example >>> applications, highlight best practices for deploying Hadoop in your >>> environment, discuss plans for the future of the technology, and >>> provide pointers to the many resources available for learning more. >>> >>> In addition to providing more information about the Hadoop >>> platform, a major goal of this talk is to begin a dialogue with the >>> ATLAS research team on how the tools commonly used in their >>> environment compare to Hadoop, and how Hadoop could improve better >>> to serve the high energy physics community. >>> >>> Short Biography: Jeff Hammerbacher is Vice President of Products >>> and Chief Scientist at Cloudera. Jeff was an Entrepreneur in >>> Residence at Accel Partners immediately prior to founding Cloudera. >>> Before Accel, he conceived, built, and led the Data team at >>> Facebook. The Data team was responsible for driving many of the >>> applications of statistics and machine learning at Facebook, as >>> well as building out the infrastructure to support these tasks for >>> massive data sets. The team produced two open source projects: >>> Hive, a system for offline analysis built above Hadoop, and >>> Cassandra, a structured storage system on a P2P network. Before >>> joining Facebook, Jeff was a quantitative analyst on Wall Street. >>> Jeff earned his Bachelor's Degree in Mathematics from Harvard >>> University and recently served as contributing editor to the book >>> "Beautiful Data", published by O'Reilly in July 2009. >>> >>> The InDiCo link is here: http://indico.cern.ch/conferenceDisplay.py?confId=59791 >>> >>> With regards, >>> >>> David Myers. >>> >>> -------------- European Organization for Nuclear Research >>> -------------- >>> Dr. D.R. Myers, Tel: +41 22 767 4646 >>> Head, Computing Security, Mobile: +41 76 487 3994 >>> CERN, IT Division, >>> CH-1211 Geneva 23, Switzerland. EMail: David.Myers(a)cern.ch >>> ------------------------------------------------------------------------ >>> >>> >>>

1 0

[Ct-grid] Next BiG Grid Meeting: Monday 17 August, 13:30h in H3.31
by Dennis van Dok 14 Aug '09

14 Aug '09

The Next BiG Grid meeting will be held on Monday 17 August, from 13.30-14.30h in room H3.31. Agenda: - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutes of the previous meeting(s) are located here: https://www.nikhef.nl/pub/departments/ct/grid/2009/ I wish you a joyous weekend, Dennis van Dok -- D.H. van Dok :: Software Engineer :: www.nikhef.nl :: www.vl-e.nl Phone +31 20 592 22 28 :: http://www.nikhef.nl/~dennisvd/

1 0

[Ct-grid] koffie en taart in H122 om 10:00
by Kees Huyser 13 Aug '09

13 Aug '09

Omdat ik tegenwoordig steeds jonger wordt (ik komt nl. elk jaar verder van mijn pensioen af!) vier ik dit met koffie en taart in H122 om 10:00. Je bent van harte welkom! --kees

1 0

[Ct-grid] REMINDER: next BiG Grid Meeting: Monday 3 August 2009 13.30h in room H3.31
by Dennis van Dok 01 Aug '09

01 Aug '09

The Next BiG Grid meeting will be held on Monday 3 August 2009, from 13.30-14.30h in room H3.31. Agenda: - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutes of the previous meeting(s) can be found at: https://www.nikhef.nl/pub/departments/ct/grid/2009/ Enjoy the weekend, Dennis van Dok

1 0

[Ct-grid] Next BiG Grid Meeting: Monday 20 July 2009 13.30h in room H3.31
by Oscar Koeroo 20 Jul '09

20 Jul '09

The Next BiG Grid meeting will be held on Monday July 20th 2009 (today), from 13.30-14.30h in room H3.31. Agenda: - Operations summary - Middleware - Applications - AOB Videoconf is available as usual. Minutus of the previous meeting(s) are located here: https://www.nikhef.nl/pub/departments/ct/grid/2009/ Hope to see you all, Oscar Koeroo

1 0

[Ct-grid] New VLET Release 1.0.1
by Piter T. de Boer 15 Jul '09

15 Jul '09

********************************************************* Announcement: New VLET Release: 1.0.1 (15 - July - 2009) ********************************************************* Hello users, developers and other interested people, (Apologies if you receive this email more then once). There is a new VLET Release available. Version: 1.0.1 This is a bug fix release for 1.0.0. I attached the release notes. Downloads, see: - https://gforge.vl-e.nl/frs/?group_id=19 Happy Browsing, Piter T. de Boer ****************************************************************************** VL-e Toolkit Release Notes ****************************************************************************** * * Copyright 2007-2009 The Virtual Laboratory for e-Science (VL-e) * * Licensed under the Apache License, Version 2.0 (the "License"). * You may not use this file except in compliance with the License. * You can obtain the Apache License at the following location: * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * * Info: * See: http://www.vl-e.nl/vbrowser/ * === Release notes for 1.0.1 === Date: 15 july 2009 This is a bug fix release. About the release: === - vlet-1.0.zip contains VLET "Core" without application plugins. - vlet-all-1.0.zip contains VLET with the VLEMED and AID plugins. - vlet-all-gars-1.0.zip contains the GAR files (grid services). Note that the LFC Grid Service is NOT needed anymore to access LFC servers. Updates: === - UserGuide has been updated to cover most of the significant changes in 1.0 Fixed bugs: === - The value of 'replicasNrOfTries' was always reset to 0. This has been fixed. - Replicas from SRM V1 Storage Elements can now be read. Storing at a V1 SRM service is still not supported as most storage element support a V2.2 interface as well which will be used if available. - Under certain conditions uploading a file to an LFC location would try the upload only once instead of the specified 'replicaNrOfTries'. - A bug in the parsing or user and domain settings of an SRB location has been fixed which garbled the SRB Server Settings. - Under Cygwin the local filesystem implementation would try to use the 'stat' command which failed. This has been disabled. - When editing a text using the embedded text editor, the original file permissions would be reset. These are now kept. - Port numbers have been removed from replicas registered in the LFC server to stay compatible with the lcg-* commands. - As of VLET 1.0, all JavaGAT libraries and dependencies have been removed. It wasn't never used and no plugins ever have been developed using JavaGAT. - See bugtracker for more details about the above bugs. === Release notes for 1.0 === Date: 27 May 2009 Features: === VBrowser: - Added monitoring (still work in progress). Some methods now will display a 'Task Monitor' similar to the already existing "Transfer Dialog". - New Properties editor. Shows resource properties in separate tabs. VRS/VFS: - New API. See vrs.* and vfs.* classes. - SSH Passphrase authentication added ! Use ~/.ssh/id_rsa file or change it from the SSH option to point to your SSH key needed for passphrase authentication. - LFC: Added New Pure Java implementation for LFC - BDII Service is now used to query VO allowed Storage Elements. Replica URIs are automatically matched against these Storage Elements. - Old 'WrapperLFC' is not supported anymore. The grid service has been removed from the gar distribution. - Updated SRM driver to work correctly and efficient with LFC. - Third party copying between GridFTP,SRM and LFC has been updated to work with all possible combinations between them, since the default transport protocol used in SRM and LFC is GridFTP. - Added transfer monitoring in 3rd party GridFTP transfers. uricopy.sh: - added "-info" and "-mkdirs" option to uricopy.sh script. The latter create the full path of the target directory if not existing yet. Script tested with new LFC driver. Seems to work. Other: - VLTerm supports more vt100 and xterm codes and is now an almost working terminal emulator. - Image viewer support zoom: double click maximize/minimize and use mouse wheel to zoom in/out. API: - Updated Virtual Resource/FileSystem API to "1.0" See JavaDOCs for details. Bugs: === - Resource Folders now correctly save a multilevel hierarchy of Resource Descriptions. - Default SRB and LFC properties were not correctly read from vletrc.prop. Use "srb.<server property>=..." and lfc.<server property>=..." to specify installation defaults. - Fixed a lot a bugs due to asynchronous browsing. VBrowser panels now can be populated in the background by for example a Query Plug-in without corrupting the internal VBrowser data structure. - Fixed DataChannel Authentication mismatch when using third party transfer between old (1.x) Grid FTP servers and new (2.x). Needed for true 3rd party copying between GFTP, SRM and LFC servers. - For more bug fixed see the bugtracker.

1 0

[Ct-grid] getalletjes
by Drenth, Eduard 08 Jul '09

08 Jul '09

Hi all, For a presentation I will present a slide about the grid (see attachment), it would be nice to have some figures on the amount of machines involved where applicable. Can someone provide these figures? If possible before tomorrow afternoon? part | amount --------------------------- UI | WMS | CE | HN | WN | LFC | SE | Thanks in advance, Eduard ------------------------------------------------ Eduard Drenth Logica Groningen Java / XML specialist 06-20943428 Please help Logica to respect the environment by not printing this email / Pour contribuer comme Logica au respect de l'environnement, merci de ne pas imprimer ce mail / Bitte drucken Sie diese Nachricht nicht aus und helfen Sie so Logica dabei die Umwelt zu schuetzen / Por favor ajude a Logica a respeitar o ambiente nao imprimindo este correio electronico. This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.

3 2

← Newer
1
...
11
12
13
14
15
16
17
...
25
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

ct-grid