Friday, January 25, 2013

Sharing

As we try and present this underlying complexity in interfaces and high-level API, certain themes  keeps repeating.  iRODS provides a fairly comprehensive set of tools to manage complex collections, including metadata management, automatic system metadata maintenance,  a global logical namespace, policy enforcement, ACL management, audit, and the like.

iRODS has a full ACL management system.  This includes the ability to federate iRODS zones and allow cross-zone access right management.  It's very powerful when combined with the audit log capability of the iRODS server.  iRODS has the mechanics covered...how can we take these elements and combine them in ways that create new services?  That's part of the fun of working on Jargon and iRODS user interfaces.  One thing that intrigues me is to look at iRODS as an application development platform from which new kinds of policy aware, distributed data applications are formed.  Sometimes the point is to look at merging iRODS middleware with other services (like Sesame and Lucene, as we're working on with the HIVE integration project, something we'll look at later).

A basic goal of Jargon and the iDrop projects is to create API and interfaces that can present iRODS as a familiar cloud-based data store, enhancing find-ability via social, metdata, and search capabilities.  We've leveraged iRODS AVU metadata to implement familiar 'tagging' as well as 'starred' folders and collections.  Those are reflected in the iDrop2 web interface, and we've done a few demos of this in prior posts.


Discovering and depicting shares has been a common use case.  In a way, you could do this by just querying collections or files in relation to their ACL settings, developing a list of shared files and collections.  The issue some had encountered is that iRODS collections can be BIG...lots of files, lots of deeply nested collections.  The queries get big, and the results that a user would get become incomprehensible in an interface.  The answer that we've cooked up with a few community folks is to develop a 'share' as a first class object.  That sounds fancy, but all it means is that, instead of treating a deeply nested collection of 10,000 files as 10,000 shared objects, why not just treat the parent collection as the share, and portray the contents of this share as a 'mount' you can make in an interface.  This way, you see 10 shares with meaningful names, instead of a pagable list of 5,000 files.  We just mixed the existing ACL system with  a new AVU entry very much like the starred folder.

So now, shares work like this:


  • Find a file or collection
  • Mark as 'shared'
    • set inheritance on collections
    • select users and the access rights you want
    • use the sharing service to mark as a share
  • Display the shares
    • We're using specific (SQL) query to develop more complex SQL queries to discover things like:
      • collections I'm sharing with others
      • collections shared with me
      • we'll add things like specifying users involved in the shares as we go
  • In iDrop web, you can now get a quick view (below) that can show things like 'files shared with me'.



As you can see, the 'home' landing page is presenting some of these 'overlays' on the AVU system.  In this case, the user has selected 'Folders shared with me", and the sharing service in Jargon we'll talk about runs the queries to inspect AVU metadata and ACL information to derive the list of shares.  Note that these are at the top level, and are portrayed using an alias, which is a share name that you give when the share is created.   

Note that clicking on the folder icon then sets that share as the top of the tree in the browse view, so a share becomes a 'mount' or a 'drive'.  The nice thing is that shares then get out of the way, and you are back to normal iRODS ACLs and collections, without a lot of cruft.  Here's what you see when you click on a folder in the share view...





What's cool is how unremarkable all of that is!  Note that the browse tree is looking into the share.  I've revealed the metadata tab so you can see the special iRODSUserTagging:Share tag that denotes the share.  Once a share is established, you can alter the rights to the share by just editing ACLs, so if the ACL is removed for a user, their share disappears.  I need to add a share button to this interface, and then we can show how to establish the share.  The code is done though...we'll look at that.

Code for sharing


Sharing is part of jargon-core, in the jargon-user-tagging subproject.  This is in the development branch of git, and as 3.3.0-SNAPSHOT, as of this writing.

Inside the org.irods.jargon.usertagging.sharing package, the key interface is the IRODSSharingService, which has the following interface:


package org.irods.jargon.usertagging.sharing;

import java.util.List;

import org.irods.jargon.core.exception.DataNotFoundException;
import org.irods.jargon.core.exception.FileNotFoundException;
import org.irods.jargon.core.exception.JargonException;
import org.irods.jargon.usertagging.domain.IRODSSharedFileOrCollection;
import org.irods.jargon.usertagging.domain.ShareUser;

/**
 * Service interface to create and manage shares. Like the star and tagging
 * facility, a share is a special metadata tag on a Collection or Data Object,
 * naming that item as shared. In the process of declaring the share, the proper
 * ACL settings are done.
 * * Sharing using a special tag at the 'root' of the share avoids representing
 * every file or collection in a deeply nested shared collection as 'shared', as
 * it would be based purely on the ACL settings. As a first class object, a
 * share can have an alias name, and is considered one unit.
 * 
 * @author Mike Conway - DICE (www.irods.org)
 * 
 */
public interface IRODSSharingService {

 /**
  * Create a new share. This will tag the top level as a shared collection
  * and data object, and set requisite AVUs for the provided set of users.
  * Note that collections will have recursive set of permissions, as well as
  * inheritance.
  * 
* Note that the share is only settable as originating from the file or * collection owner * * @param irodsSharedFileOrCollection * {@link IRODSSharedFileOrCollection} representing the share * @throws ShareAlreadyExistsException * if a share has already been defined * @throws FileNotFoundException * if the absolute path does not exist in iRODS * @throws JargonException */ void createShare(IRODSSharedFileOrCollection irodsSharedFileOrCollection) throws ShareAlreadyExistsException, FileNotFoundException, JargonException; /** * Given an absolute path to an iRODS file or collection, return the * IRODSSahredFileOrCollection that may exist. Note that * null is returned if no such share exists, and a * FileNotFoundException is returned if the absolute path does * not exist. * * @param irodsAbsolutePath * String with a valid iRODS absolute path to a file * or collection * @return {@link IRODSSharedFileOrCollection} or null * @throws FileNotFoundException * if the target absolute path does not exist in iRODS * @throws JargonException */ IRODSSharedFileOrCollection findShareByAbsolutePath(String irodsAbsolutePath) throws FileNotFoundException, JargonException; /** * Remove the share indicated at the given absolute path. Note that this method will silently ignore an occasion where a share does not * exist for the given path. *
* NOTE: an outstanding issue remains, which is how to handle the ACLs associated with the given file or collection. Right now the share goes away, * but the ACLs remain. It is under consideration to remove all ACLs, or add a flag or method variant that will either preserve or delete the associated * ACLs. * * @param irodsAbsolutePath String with a valid iRODS absolute path to a file * or collection * @throws FileNotFoundException if the iRODS absolute path does not point to a file or collection * @throws JargonException */ void removeShare(String irodsAbsolutePath) throws FileNotFoundException, JargonException; /** * Retrieve a list of collections shared by the given user and zone. No shares will return an empty set. *
* Note here that, for efficiency, the list of users (via theACLs) is not returned in this variant. It is intended that obtaining * the listing would be done as a separate request. A variant may be added later that does do this extra processing * @param userName String with the name of the user who is doing the sharing, based on the owner of the collection. * @param userZone String with the zone for the user. This may be set to blank, in which case the zone of the * logged in user will be used *
* Note that this method uses Specific Query, and the listSharedCollectionsOwnedByUser query alias must be provided. This can * be initialized by running a script in the jargon-user-tagging project to set up all required specific queries. See project documentation. * This method requires and iRODS server that supports Specific Query (iRODS 3.1+) * @return List of {@link IRODSSharedFileOrCollection} that is shared by the user * @throws JargonException */ List listSharedCollectionsOwnedByAUser( String userName, String userZone) throws JargonException; /** * Retrieve a list of collections shared with a given user by another user, as determined by the owner of that collection. *
* Note here that, for efficiency, the list of users (via theACLs) is not returned in this variant. It is intended that obtaining * the listing would be done as a separate request. A variant may be added later that does do this extra processing * @param userName String with the name of the user who is doing the sharing, based on the owner of the collection. * @param userZone String with the zone for the user. This may be set to blank, in which case the zone of the * logged in user will be used *
* Note that this method uses Specific Query, and the listSharedCollectionsSharedWithUser query alias must be provided. This can * be initialized by running a script in the jargon-user-tagging project to set up all required specific queries. See project documentation. * This method requires and iRODS server that supports Specific Query (iRODS 3.1+) * @return List of {@link IRODSSharedFileOrCollection} that is shared by a party with the user * @throws JargonException */ List listSharedCollectionsSharedWithUser( String userName, String userZone) throws JargonException; /** * Handy method to retrieve ACL share details for a share at the given absolute path. Note that if * there is no share, an empty list is returned. This seems to convey the message with the least amount of surprise. * * @param String with a valid iRODS absolute path to a file * or collection * @return List of {@link ShareUser} * @throws FileNotFoundException if the path cannot be found * @throws JargonException */ List listUsersForShare(String irodsAbsolutePath) throws FileNotFoundException, JargonException; /** * Update the name of the share at the given path * @param irodsAbsolutePath String with a valid iRODS absolute path to a file * or collection * @param newShareName String with the desired name of the share * @throws FileNotFoundException if the iRODS file or collection is missing * @throws DataNotFoundException if a current share is not found * @throws JargonException */ void updateShareName(String irodsAbsolutePath, String newShareName) throws FileNotFoundException, DataNotFoundException, JargonException; }








Note that there is a domain object that represents the share, the IRODSSharedFileOrCollection.  This object holds a path, an alias, and a list of users and their access rights desired for the share.  It's really that easy, you just call createShare() with the right parameters, then you call list* methods to get the shares back.  It handles ACLs, metadata, and inheritance of permissions.

The twist is that the queries are too complex for GenQuery, the built-in SQL-like syntax for querying the iRODS metadata catalog.  It uses Specific or SQL query, which is essentially like iRODS stored procedures, where you define a query with parameters via Jargon or the iadmin icommand.  We need to run a couple of shell scripts with iRODS rodsadmin privileges to enable specific query support.

First, Jargon itself has a few simple Specific query statements it relies on to support specific query, namely, it has methods to list and find specific queries stored in iRODS based on alias.  These can be set up by running the jargon-specquery.sh commands in the jargon-core library.  I think that these will soon be baked into iRODS, but as of 3.2 they are not yet there.

Next, we're in the process of developing the complex SQL required to list different sorts of shares, and these are being developed in the jargon-user-tagging subproject of jargon, and are provisioned by running the usertagging_specquery.sh.

Run those two scripts as rodsadmin and you should be good to go!  We're testing and integrating with iDrop web2, so it'll be a few weeks before the final API and queries are settled, but it's working now, and worth a try...see if it satisfies your use case!





No comments:

Post a Comment