With all of these mechanisms for controlling access to data, how can you do ad-hoc, or 'easy' sharing? If I open up files and collections to be shared, how can I manage this shared data? There are several answers to this question, and one of the most useful is to use the new ticket facility that appeared in iRODS 3.1.
We'll look at ticket support in Jargon and in iDrop web to illustrate how you can use tickets in your own projects.
Tickets are tokens, and they have interesting properties (taken from Wayne's notes on tickets):
- Only object and collection owners can create them (via Jargon or via the iticket iCommand).
- Users may access objects and collections via a ticket, even if they do not have access via the ACL system.
- Anonymous users may access data with a ticket
- Tickets have configurable limits:
- Limited to certain groups or users
- Limited lifetime
- Limited to certain DNS names
- Limited number of accesses, byte counts, file counts
- Ticket access is audited if audit trails are enabled in iRODS
So not only does a ticket provide an easy way to allow ad-hoc access, it provides extensions to control such sharing. Tickets become very useful behind webt interfaces, where iRODS files and collections may be accessed via a URL, and that access is controlled.
Jargon has support in the jargon-core library. The jargon-ticket sub-project has services for managing as well as using tickets.
Ticket Administration
As mentioned, there are new commands in iticket to do the administration of tickets. In interfaces, one may use the TicketAdminService in the org.irods.jargon.ticket package. Here is a sample of some of the methods available in the TicketAdminService:
/** * Create a ticket for access to iRODS * * This operation may be done on files or collections. Note that, for * collections, the inheritance bit will be set, so that the ticket creator * may have permissions on any files that grantees create in the collection. * * @param TicketCreateModeEnum * mode ticket create mode - read or write * @param IRODSFile * file existing IRODS file or collection * @param String * ticketID used to specify ticket key to be used for this ticket * @throws JargonException * */ String createTicket(TicketCreateModeEnum mode, IRODSFile file, String ticketId) throws JargonException; /** * Delete a ticket for access to iRODS * * @param String * ticketID used to specify ticket key to be deleted * @returnbooleanthat will betrueif the ticket * was found to delete.falsemeans that the delete was * not successful, due to the ticket not being found. This can be * ignored. * @throws JargonException * */ boolean deleteTicket(String ticketId) throws JargonException; /** * Generate a list of all tickets for data objects (files). Note that, for a * regular user, this will be tickets for that user. For a rodsadmin, this * will be all tickets. * * @param ticketId * - string used to identify the ticket * @return {@link Ticket} object for specified ticket string identifier * @throws DataNotFoundException * if ticket cannot be found * @throws JargonException */ Ticket getTicketForSpecifiedTicketString(String ticketId) throws DataNotFoundException, JargonException;
There are lots more methods there, you can see the full listing of the service here. Using that service, one may create, list, and manage tickets, and set various restrictions. Tickets themselves are represented by a POJO domain object that contains information on these restrictions, and on current reported usage.
Using Tickets
Once tickets are configured on the system, the ticket string is your token to access iRODS data. The jargon-ticket library has a TicketClientOperations service that can be used to redeem a ticket for get and put operations. For those using the DataTransferOperationsAO code in jargon-core, these methods should look familiar. They provide parameters to define the source and target of the operations, as well as a callback listener for transfer events, and a TransferControlBlock to communicate to the transferring process, like so:/** * Wraps a put operation with ticket semantics. * * Put a file or collection to iRODS. Note that 'force' is not supported * with tickets at this time, so overwrites will return an *OverwriteException* * @param ticketString *Stringwith the unique ticket string * @param irodsSourceFile * {@link org.irods.jargon.core.pub.io.IRODSFile} that points to * the file or collection to retrieve. * @param targetLocalFile *Filethat will hold the retrieved data. * @param transferStatusCallbackListener * {@link org.irods.jargon.core.transfer.TransferStatusCallbackListener} * implementation that will receive callbacks indicating the * real-time status of the transfer. This may be set to null if * not required * @param transferControlBlock * an optional * {@link org.irods.jargon.core.transfer.TransferControlBlock} * that provides a common object to communicate between the * object requesting the transfer, and the method performing the * transfer. This control block may contain a filter that can be * used to control restarts, and provides a way for the * requesting process to send a cancellation. This may be set to * null if not required. * @throws OverwriteException * if an overwrite is attempted and the force option has not * been set * @throws DataNotFoundException * if the source iRODS file does not exist * @throws JargonException */ void putFileToIRODSUsingTicket( final String ticketString, final File sourceFile, final IRODSFile targetIrodsFile, final TransferStatusCallbackListener transferStatusCallbackListener, final TransferControlBlock transferControlBlock) throws DataNotFoundException, OverwriteException, JargonException;
Note here that the ticket string is presented, and will be processed using the IRODSAccount used to connect. This IRODSAccount may be the anonymous user, and that is the typical case currently supported in iDrop web.
Tickets do have a limitation for get and put operations, they do not operate on streaming operations. Unfortunately, streaming is the common operation used for browser uploads and downloads. To get around this limitation, the jargon-ticket library adds methods to achieve streaming uploads and downloads using temporary intermediate files. This is typically how HTTP uploads work under the covers in Tomcat, so that does not seem especially onerous. So in the TicketClientOperations, you will find methods like this:
Tickets do have a limitation for get and put operations, they do not operate on streaming operations. Unfortunately, streaming is the common operation used for browser uploads and downloads. To get around this limitation, the jargon-ticket library adds methods to achieve streaming uploads and downloads using temporary intermediate files. This is typically how HTTP uploads work under the covers in Tomcat, so that does not seem especially onerous. So in the TicketClientOperations, you will find methods like this:
/** * Wraps a get operation with ticket semantics. * * Get a file or collection from iRODS to the local file system. This method * will detect whether this is a get of a single file, or of a collection. * If this is a get of a collection, the method will recursively obtain the * data from iRODS. * * @param ticketString *Stringwith the unique ticket string * @param irodsSourceFile * {@link org.irods.jargon.core.pub.io.IRODSFile} that points to * the file or collection to retrieve. * @param targetLocalFile *Filethat will hold the retrieved data. * @param transferStatusCallbackListener * {@link org.irods.jargon.core.transfer.TransferStatusCallbackListener} * implementation that will receive callbacks indicating the * real-time status of the transfer. This may be set to null if * not required * @param transferControlBlock * an optional * {@link org.irods.jargon.core.transfer.TransferControlBlock} * that provides a common object to communicate between the * object requesting the transfer, and the method performing the * transfer. This control block may contain a filter that can be * used to control restarts, and provides a way for the * requesting process to send a cancellation. This may be set to * null if not required. * @throws OverwriteException * if an overwrite is attempted and the force option has not * been set * @throws DataNotFoundException * if the source iRODS file does not exist * @throws JargonException */ void getOperationFromIRODSUsingTicket( final String ticketString, final IRODSFile irodsSourceFile, final File targetLocalFile, final TransferStatusCallbackListener transferStatusCallbackListener, final TransferControlBlock transferControlBlock) throws DataNotFoundException, OverwriteException, JargonException; /** * Given an iRODS ticket for a data object, return an object that has an *InputStreamfor that file, as well as the length of data to * be streamed. This method is oriented towards applications that need to * represent the data from iRODS as a stream. * * Note that currently only 'get' and 'put' are supported via tickets, so * mid-tier applications that wish to stream data back to the client need to * do an intermediate get to the mid-tier platform and then stream from this * location. * * Tickets are limited in what they can access, so various operations that * refer to the iCAT, such as obtaining the length, or differentiating * between a file and a collection, cannot be done in the typical way. As a * work-around, this object holds the lenght of the cached file so that it * may be sent in browser responses. * * @param ticketString *Stringwith the unique string that represents the * ticket * @param irodsSourceFile * {@link IRODSFile} that represents the data to be streamed back * to the caller * @param intermediateCacheRootDirectory * {@link File} on the local file system that is the directory * root where temporary files may be cached. Note that, upon * close of the returned stream, this file will be cleaned up * from that cache. * @return {@link FileStreamAndInfo} with a buffered stream that will delete * the cached file upon close. This object also contains a length * for the file. * @throws DataNotFoundException * if the ticket data is not available * @throws JargonException */ FileStreamAndInfo redeemTicketGetDataObjectAndStreamBack( String ticketString, IRODSFile irodsSourceFile, File intermediateCacheRootDirectory) throws DataNotFoundException, JargonException; /** * This method specifically addresses 'upload' scenarios, where data is * supplied via anInputStream, representing the contents that * should be placed in a target file with a givenfileName* underneath a given target iRODS collection path in *irodsCollectionAbsolutePath. This method will take the * contents of the input stream, store in a temporary cache location as * described by theintermediateCacheRootDirectory, then put * that file to iRODS. Once the operation is complete, the temporary file * will be removed. This removal is done in a finally block, so that if the * put operation fails, it should minimize leakage of old files. * * The primary use case for this method is in mid-tier applications where a * file is being uploaded from a browser. Since the iRODS ticket system does * not support input or output streams, the upload needs to be wrapped to * emulate a direct streaming via a ticket. * * @param ticketString *Stringwith the unique ticket id, which must have * write privilages * @param irodsCollectionAbsolutePath *Stringwith the target iRODS parent collection * absolute path. The file will be placed under this collection * using the givenfileName* @param fileName *Stringwith the name of the file being uploaded * to iRODS * @param inputStreamForFileData *InputStreamwhich should be properly buffered by * the caller. This could be the input stream resulting from an * http upload operation * @param temporaryCacheDirectoryLocation * {@link File} representing a temporary local file system * directory where temporary files may be cached * @throws DataNotFoundException * if the ticket information is not available * @throws OverwriteException * if an overwrite would occur * @throws JargonException */ void redeemTicketAndStreamToIRODSCollection(String ticketString, String irodsCollectionAbsolutePath, String fileName, InputStream inputStreamForFileData, File temporaryCacheDirectoryLocation) throws DataNotFoundException, OverwriteException, JargonException;
Example 1 - Create a ticket for a file and download it
Using just the TicketAdminService, and the TicketClientOperations classes, you can add tickets, and then get and put data to iRODS. The JUnit tests in jargon-ticket are a great resource to see the ticket API in use. The TicketClientOperationsImplTest has a good example. Here's a snippet that will take an existing file, create a ticket on it, and then get the file:
// put a read ticket on the file
TicketAdminService ticketSvc = new TicketAdminServiceImpl(
irodsFileSystem.getIRODSAccessObjectFactory(), irodsAccount);
ticketSvc.deleteTicket(testFileName);
ticketSvc.createTicket(TicketCreateModeEnum.READ,
destFile, testFileName);
IRODSFile getIRODSFile = irodsFileFactory
.instanceIRODSFile(targetIrodsFile);
File getLocalFile = new File(absPath + "/" + testRetrievedFileName);
getLocalFile.delete();
// now get the file as secondary user with ticket
IRODSAccount secondaryAccount = testingPropertiesHelper
.buildIRODSAccountFromSecondaryTestProperties(testingProperties);
TicketClientOperations ticketClientService = new TicketClientOperationsImpl(
irodsFileSystem.getIRODSAccessObjectFactory(), secondaryAccount);
ticketClientService.getOperationFromIRODSUsingTicket(testFileName,
getIRODSFile, getLocalFile, null, null);
In the above example, a TicketAdminService is created. This takes a reference to an IRODSAccessObjectFactory, and the IRODSAccount that is the owner of the file to be shared via a ticket. Since this is a unit test, it deletes and then re-creates the ticket, based on an arbitrary ticket string.
To test the newly created ticket, A TicketClientOperations service is created, with the same sorts of parameters used to create the TicketAdminService. In the case of this snippet, a secondaryIrodsAccount variable is created. In the test case, this secondary account is indeed an anonymous user.
Finally, the ticketClientService.getOperationFromIRODSUsingTicket() method is called as anonymous. Really, it's that simple, it's a normal get operation, with the added ticket semantics!
Tickets in the Mid-Tier
We can use iDrop web as an example of tickets in the mid-tier. If you are putting data onto the web, tickets are a great tool. To use them, you need to address two issues:
- How do you create a URL to redeem a ticket?
- What happens when a ticket request comes in via a URL?
Let's take a look at creating and then getting a file based on a ticket in iDrop web...
That video shows tickets in a web interface, so let's look at how to address the issues I mentioned.
The TicketDistribution object holds references to URLs with and without a landing page, as well as the iRODS URI format for the ticket file. In order to properly populate the host, port, and other information in these URLs, the caller can define a TicketDistributionContext object, which is passed to the TicketDistributionService. This snippet of code from the TicketDistributionServcieImplTest JUnit test suite shows the parts of the TicketDistributionContext, and how a TicketDistribution is obtained.
The TicketDistribution can be rendered on a web page view, or passed to any external client (even via an email), and the URL should resolve properly to a host that can process the ticket requests.
In the future, we can 'dress up' the landing page, add the ability to call a URL shortening service, and add a mailer directly from iDrop.
The 'context' path that you give to the TicketDistributionContext is just the action name and additional path information that points to a controller or CGI process. It's up to the controller or process at the receiving end to use the TicketClientOperations service to handle the streaming of the iRODS data via HTTP. For the iDrop demo in the video, here's the 'redeemTicket' method of the TicketAccessController:
And that's how you use tickets. It's much more complex and capable than this example, but I suspect that read tickets on files will be the vast majority of use cases.
Defining URLs for Shared Data
The jargon-ticket library has a service that can help produce ticket URLs. One may look up a ticket from the TicketAdminService, and then ask the TicketDistributionService to get the TicketDistribution using this method:/**
* Get information about ticket distribution channels for a given valid
* iRODS ticket
*
* @param ticket
* {@link Ticket} for which distribution information will be
* generated
* @return {@link TicketDistribution} with extended information on accessing
* the given ticket
* @throws JargonException
*/
TicketDistribution getTicketDistributionForTicket(final Ticket ticket)
throws JargonException;
The TicketDistribution object holds references to URLs with and without a landing page, as well as the iRODS URI format for the ticket file. In order to properly populate the host, port, and other information in these URLs, the caller can define a TicketDistributionContext object, which is passed to the TicketDistributionService. This snippet of code from the TicketDistributionServcieImplTest JUnit test suite shows the parts of the TicketDistributionContext, and how a TicketDistribution is obtained.
String host = "localhost";
int port = 8080;
boolean ssl = true;
String context = "/idrop-web/tickets/redeemTicket";
IRODSAccount irodsAccount = testingPropertiesHelper
.buildIRODSAccountFromTestProperties(testingProperties);
IRODSAccessObjectFactory irodsAccessObjectFactory = Mockito
.mock(IRODSAccessObjectFactory.class);
TicketServiceFactory ticketServiceFactory = new TicketServiceFactoryImpl(
irodsAccessObjectFactory);
TicketDistributionContext ticketDistributionContext = new TicketDistributionContext();
TicketDistributionService ticketDistributionService = ticketServiceFactory
.instanceTicketDistributionService(irodsAccount,
ticketDistributionContext);
ticketDistributionContext.setContext(context);
ticketDistributionContext.setHost(host);
ticketDistributionContext.setPort(port);
ticketDistributionContext.setSsl(ssl);
Ticket ticket = new Ticket();
ticket.setTicketString("xxx");
ticket.setIrodsAbsolutePath("/yyy");
TicketDistribution ticketDistribution = ticketDistributionService
.getTicketDistributionForTicket(ticket);
The TicketDistribution can be rendered on a web page view, or passed to any external client (even via an email), and the URL should resolve properly to a host that can process the ticket requests.
In the future, we can 'dress up' the landing page, add the ability to call a URL shortening service, and add a mailer directly from iDrop.
Defining URLs for Shared Data
The 'context' path that you give to the TicketDistributionContext is just the action name and additional path information that points to a controller or CGI process. It's up to the controller or process at the receiving end to use the TicketClientOperations service to handle the streaming of the iRODS data via HTTP. For the iDrop demo in the video, here's the 'redeemTicket' method of the TicketAccessController:
/**
* Use a direct data url to stream back data for a ticket
*/
def redeemTicket = {
log.info("redeemTicket()")
def ticketString = params['ticketString']
if (ticketString == null) {
throw new JargonException("no ticketString passed to the method")
}
def irodsURIString = params['irodsURI']
if (irodsURIString == null) {
throw new JargonException("no irodsURI parameter passed to the method")
}
log.info("ticketString: ${ticketString}")
log.info("irodsURIString: ${irodsURIString}")
def useLandingPage = params['landingPage']
if (useLandingPage) {
log.info("reroute to landing page")
redirect(action:"landingPage", params:params)
return
}
// get an anonymous account based on the provided URI
//URI irodsURI = new URI(URLDecoder.decode(irodsURIString))
String mungedIRODSURI = irodsURIString.replaceAll(" ", "&&space&&")
URI irodsURI = new URI(mungedIRODSURI)
String filePath = irodsURI.getPath()
log.info("irodsFilePath:${filePath}")
filePath = filePath.replaceAll("&&space&&", " ")
String zone = MiscIRODSUtils.getZoneInPath(filePath)
log.info("zone:${zone}")
IRODSAccount irodsAccount = IRODSAccount.instanceForAnonymous(irodsURI.getHost(),
irodsURI.getPort(), "", zone,
"")
File tempDir =servletContext.getAttribute("javax.servlet.context.tempdir")
log.info("temp dir:${tempDir}")
TicketClientOperations ticketClientOperations = ticketServiceFactory.instanceTicketClientOperations(irodsAccount)
IRODSFile irodsFile = irodsAccessObjectFactory.getIRODSFileFactory(irodsAccount).instanceIRODSFile(filePath)
FileStreamAndInfo info = ticketClientOperations.redeemTicketGetDataObjectAndStreamBack(ticketString, irodsFile, tempDir)
log.info("got input stream, ready to pipe")
def length = info.length
log.info("file length = ${length}")
log.info("opened input stream")
response.setContentType("application/octet-stream")
response.setContentLength((int) length)
response.setHeader("Content-disposition", "attachment;filename=\"${irodsFile.name}\"")
response.outputStream << info.inputStream // Performing a binary stream copy
}
In the example, the parameters are picked apart, and an anonymous IRODSAccount is created from the information in the irodsURI parameter of the URL. The TicketClientOperations is used to obtain an output stream from the iRODS file, and that output stream is read and piped back to the client browser.And that's how you use tickets. It's much more complex and capable than this example, but I suspect that read tickets on files will be the vast majority of use cases.
No comments:
Post a Comment