Hello Reader, here is this month’s iRODS news and developments!

If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere, I’ll do my best to let you know. Or just drop me a mail to say Hi. Always nice to hear from people, particularly in these pandemic times!

I’d love your thoughts and feedback on how this newsletter could be better for you.

News

More repos! Thanks to the newsletter subscriber who suggested this. I’ve done a big refactor of the code that generates this newsletter, and will now report on changes in the main, Python iRODSClient, NFSRods and icommands repositories.

I’ve had a request to also cover Yoda - what do you think, is that diluting the newsletter, or enhancing it? I’m minded to do so since I’m working with it in my day job, but what do YOU think? Hit reply and let me know!

​iRODS Development Update - March 2021​

iRODS Internship: Summer 2021. The iRODS Consortium is looking for qualified, motivated candidates for a paid, full-time, remote, summer internship. This internship is open to international students but may require some extra paperwork. A US Bank account is highly recommended if you are not in the United States.

User Group Meeting

Registration for the 13th Annual #iRODS User Group Meeting is open! Hosted virtually by Sanger Institute, the meeting will take place from June 8 - 11.

​Registration is open for this year’s virtual iRODS UGM ​

Want the chance to present a 20-minute talk or 40-minute demo at the iRODS Virtual UGM? Submit an abstract by May 15! Authors of accepted talks + demonstrations will be invited to present at the User Group Meeting https://irods.org/ugm2021/​

Main Repository Activity

Open Issues

​Investigate locking objects BEFORE creating new replicas​

Part of the work for 4.2.9 towards Logical Locking​

​imeta addw gives non-zero error code on successful upload​

Oops, duplicate​

​microservices fail when called from pep_database_close_pre​

Worth reading if you use, or intend to use, pep_database_close_pre.

​irods::plugin_context not serialized in pep_database_close_* PEPs​

As above.

​iRODS rule language parser(?) logging “Error updating cache.” ​

Epic work here from Tony, to quote;

“I copy into /etc/irods a 17k line rule base that provides an implementation for every dynamic PEP in stock iRODS 4.2.8”

It seems there is a max 30MB rule execution time limit in the code.

​irodHost is probably not set correctly status = -302000 USER_RODS_HOST_EMPTY​

The RENCI folks are trying to be very helpful here, but I do feel that the user who raised this probably should have been talking to their local support first… :-)

Also, 3.3.1! I just retired the last of our 3.3.1 systems this year!

​isysmeta cannot modify anything​

Working on a reproducer, it seems…

​iRODS Should Handle Daylight Savings Time Changes without issues​

TL;DR if you copy around your .irodsA file, be wary of timezone changes.

​ips should have a way to display the hostnames in its output​

Currently, it’s just IP addresses, so a flag to do DNS resolution would be helpful, and potentially illuminating sometimes!

​Object remains in intermediate state when close operation returns an error.​

Part of the work for 4.2.9 towards Logical Locking​

​On leaf-of-tree resource which is down, imv / irm succeed with error​

​maximum_number_of_concurrent_rule_engine_server_processes seems not to work​

Should be fixed in 4.2.9?

​optimization/atomicity when manipulating resource hierarchies​

TL;DR perform multiple tree surgery simultaneously. Helpful in stopping failures during the moment of resource migration, which will only be a concern if you have near-constant uploads going on (as we do), or want to pursue the ellusive goal of zero downtime.

​ibun reporting & documentation of errors handing symlinks​

Do any subscribers use ibun? How do you find it? It’s never been quite goldilocks enough for us…

​data okay, but rsync filling logs with ERROR when two providers in HA configuration (-4000)​

This reminds me, I should try and reproduce this.

​iuserinfo incorrect parsing of default username​

Different code paths for iuserinfo and iuserinfo

​Running icommands with new user throws errors but still works​

​noisy logs - caught exception when replicating to a resource which doesn’t exist​

​irm with force flag should not care about “does not exist”​

​ifsck should continue not terminate when hitting a permission error​

​resource_id information missing from izonereport json​

​iput across federation does not report CAT_NO_ACCESS_PERMISSIONs ​

​iuserinfo returning multiple results​

​iRODS doesn’t handle or report failure of one irodsServer process​

​Support for Ubuntu 20.04​

This has a follow up on a suggested procedure for building icommands on 20.04.

​add thread to irodsAgent to detect dropped socket connections​

If you use federation and sometimes processes get killed at the Federee and you use Nagios, boy is this the ticket for you! Warning contains code by me.

​prepareNonServerSharedMemory error​

​acPostProcForPut fires off also on replication​

​irepl –rlock fails with ‘file name too long’ when irepl by itself doesn’t​

​Add detached mode to unixfilesystem plugin​

Add functionality to allow multiple Consumers to look at one shared filesystem (NFS, Lustre et al)? Yes Please (not yet)!

​new api plugin - finalize data transfer​

“A new server api plugin to check integrity of completed transfer, possibly checksum, update catalog, report success.”

I.e. Part of a new multipart transfer mechanism #4336, as I understand it

​unnecessary data movement within compound resource via irepl​

​Simultaneous msiDataObjRepl and msiDataObjRename can cause inconsistencies​

​iphymv doesn’t move file in composite resource tree​

​iadmin modresc rebalance failure​

​iRODS not compatible with MySQL Connector ODBC 8.0.12 on Cen7​

​add rule name to delayed rules​

​remove iphybun​

​ifsck gives boost error when filesystem contains a directory with no permissions for irods user​

Such as the root-owned lost+found.

​Rename irodsReServer to irodsDelayServer​

​irods::resolve_plugin_path header file function needs “inline” keyword​

​imv to filename with slash does not return an error​

​Define meaning of ‘&’ in ils output in help.​

This will all be better in 4.2.9! Interim replicas will have other labels here so the lot will be documented.

Closed Issues

closed on - 2021-04-08 19:01:56 Second forced (overwrite) recursive iget places copy of collection inside previous copy of collection​

closed on - 2021-04-16 02:26:15 Setting to output debug logs only in rodsLog​

closed on - 2021-04-09 19:14:20 data object size changed to zero (0) after re-replicating modified data object with irepl​

closed on - 2021-04-09 14:27:18 replication resource does not replicate for new empty data object​

closed on - 2021-04-15 00:44:38 irepl silently ignores -S in composite resource, doesn’t update replicas​

closed on - 2021-03-01 20:05:51 iput: checksum with bulk transfer​

closed on - 2021-04-09 20:45:49 Allow irepl to handle resource hierarchies properly​

Python iRODS Client Activity

Open Issues

​Atomic operation on metadata errors with symbols in the value​

​Executing a rule requires reconnection / reauthentication to iRODS​

Closed Issues

closed on - 2021-03-18 06:55:09 Is there a way to get SSL without irods_environment.json and without .irodsA ?​

NFSRods Activity

Open Issues

​Failed to register at portmap: portmap service not available​

​