Hadn't seen this one before ...

Begin forwarded message:

From: "Ruben S. Montero" rubensm@dacya.ucm.es Date: 7 April 2009 16:12:14 GMT+02:00 To: gridway-user@globus.org Subject: [gridway-user] Fwd: [Globus] Help shape Globus' future: A call for use cases

Just in case you missed this one... and do not forget to send your GridWay use case description, needs, wish list... !

Cheers

Ruben

---------- Forwarded message ---------- From: Ian Foster foster@anl.gov Date: Tue, Mar 31, 2009 at 8:05 AM Subject: [Globus] Help shape Globus' future: A call for use cases To: announce@globus.org Cc: Raj Kettimuthu kettimut@mcs.anl.gov, cdigs-team@mcs.anl.gov

Summary Over the past few months, within the context of the NSF Community Driven Improvement of Globus Software (CDIGS: www.cdigs.org) project, we have begun re-evaluating how we should invest toward Globus' future. We believe we are starting to understand alternatives and trade-offs. Of course, our understanding of the needs of the Globus community is imperfect, and we do not want to make decisions without broader consultation. Thus, we would now like to engage with the Globus community to help us work through this evaluation. A key concern at this point in the process is that we may not have a complete enough set of uses cases that represent what people are doing with Globus, or hope to do with Globus. Thus, we ask that you help us by providing a description of how you use (or hope to use) Globus, along with contact information so that we can follow up with you to clarify details and/or participate in subsequent stages of this effort. We provide below more details on what we are looking for. Now the longer version...

Where are we? We have had tremendous successes with Globus over the past 13 years, along with some trying times. A challenging issue over the years has been adapting to the changing technology landscape. Globus Toolkit version 4 (GT4) was introduced approximately 5 years ago, and much has changed since then. So what are the circumstances that factor into our decisions on how to evolve?

Users: Many healthy, active, growing communities are relying on Globus for their day-to-day work, including TeraGrid, OSG, ESG, LIGO, caBIG, and many others. (Based on our anonymized usage reporting, it is clear there are also many more Globus users than those we know about.) These communities challenge us with the constant, healthy tension between demands for stability and new functionality. There are also new, large communities that are adopting Globus, such as BIRN and other biomedical communities, which will continue to press these sometimes conflicting demands.

Funding: We are fortunate to have significant NSF CDIGS funding for a couple more years, which allows us to focus on support and development of Globus to meet user needs. We also have a rich mix of grants for a wide range of activities, ranging from computer science research, to infrastructure deployment and support, to application integration, to complete application development. However, these efforts are of course finite in duration, and we must be constantly looking to develop new programs to carry Globus forward.

Technology trends: The technology landscape in which we operate continues to evolve at a dizzying pace, with technologies and approaches such as Cloud Computing, Software-as-a-Service (SaaS), OpenID, OpenCL, multi-core, etc. In the meantime, technology trends that we have embraced in Globus, such as SOA and Web services, continue to mature, for example with the widespread adoption of lighter-weight REST approaches.

Aging foundations: Globus relies heavily on open source components from other communities. Some component choices, such as OpenSSL, have stood the test of time. Others have not: in particular, on the Java web services front (e.g. GT4 Java Core), we find ourselves highly dependent on libraries such as Apache Axis v1.x and PureTLS, which most of the rest of the world has moved away from. Thus, not only do we have to support these technologies entirely on our own, but it becomes increasingly difficult to support our users and our own newer activities that (rightly) want to exploit new open source software libraries and approaches. These aging GT4 Java Core foundations are also inhibiting our ability to effectively address a variety of issues (e.g., scalability, REST support) in other Globus components, such as GRAM4 (aka WS-GRAM), RFT, and MDS4.

Community: The Globus Alliance (dev.globus.org) community of developers has made many valuable contributions that have seen wide adoption, such as MyProxy, GridShib, Gaards, and GridWay. However, the dev.globus.org community has not helped significantly with the sustainability of existing components by bringing new developers into to the community who add features or maintain those components.

Where do we go from here? Thanks in particular to our NSF CDIGS funding, we have some time and resources to address these issues, and we will not abruptly leave our users out in the cold now or in coming years. One CDIGS mandate is to continue supporting existing users, which we will clearly do. So regardless of what decisions are made, we will continue to support our GT4 users as long as we have demand and funding such as CDIGS to do so. But CDIGS also has two other mandates: helping new users, and working to ensure Globus' sustainability. And, of course, we have many other projects that are contributing to development of new functionality in Globus, which we can leverage as well. With all of this in mind, our intent is to evolve Globus based on three principles:

Embrace widely adopted and community supported software and standards: When GT4 Java Core was created over 5 years ago, Apache Axis 1.x was the right choice, but as discussed above the world has moved on. (You One might argue whether WSRF was the right choice at the time, but there is strong support for managing/representing state as a first class part of the web services model, as evidenced by adoption of WSRF in large communities such as caBIG, and the popularity of REST approaches throughout the Web 2.0 community.) We have begun evaluation and prototyping efforts around the many alternative web services toolkits now available, with goals of: being as conservative as possible in our choice of technologies (e.g., looking for toolkits with robust communities); being as flexible as possible in what requirements we place on our users (e.g., looking to support multiple web services toolkits); doing as little "invention" or variance from the standard technology as possible (so we do not have to ship our own modified version, and plug cleanly into existing systems); while preserving core notions that Globus users have found compelling such as rich security and first class management and representation of state.

Maintain backward compatibility as much as possible: We recognize that protocol incompatibilities have been a recurring headache for our community in the past. Sometimes these are unavoidable as one progresses forward -- particularly on the tight budget of a non-profit development group such as ours. But our decisions will be strongly influenced by our ability to maintain backward compatibility. For example, one consideration in evaluating web services toolkits is whether we can provide WSRF (GT4.0 and/or GT4.2) backward compatibility while simultaneously allowing evolution to newer SOAP- and REST-based protocols.

Address long-term sustainability: As discussed above, one key aspect of sustainability is moving Globus onto newer open source components that enjoy wide support from a much broader community than just Globus. Additionally, we have been considering: options for simplifying Globus, such as removing little-used features which present code maintenance, stability, and/or scalability issues (e.g., GRAM2&4 job rendezvous, GRAM2 stdout/err streaming); reinvesting resources into GRAM2 since it is still heavily used; ceasing additional development on some components such as GT4 Java Core, GRAM4 and RFT (e.g., only providing bug fixes, and working with users to migrate to other options); and re-implementing components in a more maintainable fashion (e.g., RFT). These examples are merely options we are considering -- at this point no firm decisions have been made.

Of particular concern is how to move forward with GT4 Java Core, and the GT4 services that are built on it. Some of our thoughts are:

GT4 Java Core: As discussed above, we are investigating various open source web services frameworks. There are three primary compatibility concerns: maintaining optional WSRF protocol compatibility while allowing for migration to non-WSRF protocols; porting existing code that uses GT4 Java Core to new frameworks; and adopting approaches that allows for easier protocol transitions in the future.

GRAM: A large part of the community is still using GRAM2, and GRAM4 (aka WS-GRAM) has sustainability issues due to its use of GT4 Java Core. We are considering re-investing in GRAM2 to solve some known issues (e.g. scalability).

RFT: While many people have expressed interest in RFT, we have had limited success with it for a variety of reasons. RFT also has sustainability issues due to its use of GT4 Java Core. We are experimenting with a replacement for RFT which is implemented using a Software-as-a-Service model, and are now looking for users interested in trying it out and giving feedback. Please contact Lisa Childers if you are interested.

GridFTP: Address some persistent usability concerns, so that GridFTP can be downloaded and operated "out of the box" with little or no configuration.

MDS4: Various communities use MDS for various purposes, including for a service registry (e.g. caBIG, Teragrid), and for monitoring services. Many good options for monitoring have emerged in the open source community in recent years, but there still appears to be a need for service registries. We are considering focusing our MDS energies more on the service registry use cases.

Security: Many communities use Globus for its powerful and flexible security (GSI). There are already activities underway for integrating GSI with other technologies such as Shibboleth, SAML, XACML, and group membership services (e.g. VOMS). However, we also need to understand the role and potential integration of GSI in the context of emerging technologies such as OpenID and OAuth.

RLS: RLS is used by several important Globus communities. We will implement a number of community-requested features, including a pluggable back end, improved logging, and more streamlined startup.

How you can help We need to hear from our users directly about how you use, or want to use, Globus. It is only through a clear understanding of these use cases that we can be confident of our decisions. What we need is a detailed description of how you use Globus, including:

what for (e.g., a science gateway, a data replication system for a science experiment, etc.), what version, what components, what features of those components (e.g., the GRAM2 features you rely on), what clients (e.g., jGlobus for GRAM and GridFTP, Condor-G with GRAM2 or 4, etc.), what sort of performance or scalability issues you are having or would like to see, etc.

Please email the following people (copied on this email) with your use cases for the following components of Globus:

GRAM 2, GRAM 4, C security: Stu Martin GridFTP, XIO: Raj Kettimuthu Reliable File Transfer (RFT): Lisa Childers GT4 Java Core, Java security: Ravi Madduri Service registries (MDS4): Lee Liming Replica Location Service (RLS): Ann Chervenak

Please include appropriate contact information, so that they can follow up for clarification or more details if necessary.

Next steps In the coming months you can expect more communication from us as we work through this activity. We will likely produce one or more technical whitepapers that describe our proposed directions, on which we will be looking for feedback. We will also produce prototypes that you can try to provide early feedback. We may also try to organize a workshop in Chicago in the near future so that we can discuss these directions in person -- please let us know if you would be interested in attending such a workshop. Viva Globus! Thanks in advance for your valuable input. The Globus Team

-- +---------------------------------------------------------------+ Dr. Ruben Santiago Montero Associate Professor Distributed System Architecture Group (http://dsa-research.org)

URL: http://dsa-research.org/doku.php?id=people:ruben Weblog: http://blog.dsa-research.org/?author=7

GridWay, http://www.gridway.org OpenNEbula, http://www.opennebula.org +---------------------------------------------------------------+