Hello Reader, here is this month’s iRODS news and developments!

If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere I’ll do my best to let you know. Or just drop me a mail to say ‘Hi’. Always nice to hear from people, particularly in these pandemic times!

I’d love your thoughts and feedback on how this newsletter could be better for you.

News

A bit late this month! My excuse is the User Group Meeting - what a few days that was!

The video’s and slides are all up from the conference at ugm link.

I’d love to know what your takeaway was from the conference - what did you learn, what, if any, key insights?

4.2.9 Released

​4.2.9 was released on 7th June​

Important news - 4.2.10

If you are planning the upgrade to 4.2.9, released just before the UGM, the you might want to hold off for 4.2.10, due in a couple of weeks.

This is due to a regression in file downloading behaviour and a separate issue with the S3 plugin.

iRODS Web Client Metalnx 2.4.0 is released.

​2.4.0 - Includes thumbnail view for Collections, and support for anonymous login to an iRODS Zone.

iRODS Python Client v1.0.0 is released!

​v1!​

Main Repository Activity

Open Issues

​failed to start irods server after compiled ​

This is on centos08 and possibly Master branch - someone likes living dangerously! :-)

​lsb_release, Python’s platform, python-distro, and /etc/os-release​

If I am reading this correctly, this is a poposed change to drop particular libraries to determine OS type and instead look at os-release file.

​Add full utf8 support to mysql​

I had assumed that it already had it, I must admit.

​nlohmann-json externals package is bad​

This affects building the externals packages, which are required to build iRODS.

​Public header naming/organization needs work​

More fruit from the work on the build system?

​release activities for 4.2.11​

Will it be in a couple of weeks? Amazing turnaround if so!

​modifying resource context via CLI results in stacktrace​

This actually relates to the S3 plugin, but it seems hard to reproduce.

​How to validate a database migration?​

Of interest to those using, or planning to use, PostgreSQL as the back end - it was put in as a ‘worth a try’ enhahcement request, however @stsnel and team at Utrecht have written just the thing - icat-database-checker!

​default_number_of_transfer_threads seems to be ignored​

A lot of discussion around acSetNumThreads and how it interacts with the defaults from the default_number_of_transfer_threads variable in server_config.json. The behaviour has changed in 4.2.9, so that it now honours the server default set that way.

​irodsctl stop always forcefully kills the local processes​

The behaviour in 4.2.x up to 4.2.7 should be that agent processes are left running to allow transfers to complete.

This bug might be saying that in 4.2.8 (and possibly 4.2.9?) the use of irodsctl to stop the server kills the processes instead of letting them drain, which would be a regression (its very useful for ‘draining’ a server for maintenance, for example).

​IRODS 4.2.9 crashes randomly at startup​

If you are using Docker you manage or test your iRODS, this might be worth a read.

​Different error codes returned for invalid user name depending on catalog service role of connected server​

Anyone use this for their workflow, I wonder? If so, reminds me of this xkcd​

​Metatool not recognizing file paths for files​

metatool is not a core iRODS tool, but comes from Bayer it seems.

​remove -Wno-ignored-qualifiers​

4-2-stable build bug

​difference in ‘schema_name’ in service account irods_environment.json​

The differences are between 4.1.x -> upgraded 4.2.x and initial 4.2.x.

​imeta command returns 0 exit code on invalid input​

There are a few of these sort of errors with imeta which has lead at least one bioinformatician of my aquintance to throw away the error codes and parse the output/check results instead, which is a shame.

Still, hopefully this will be fixed in 4.2.11?

​Define ACL meaning​

ichmod man page says that the ownership rights are ‘null’, ‘read’, ‘write’ and ‘own’, but does not define what this means, especially at the collection level. docs.irods.org likewise does not define what these ACL’s actually mean.

Bumped to 4.2.11, which is understandable given the push for 4.2.9 for UGM, but user facing documentation should always be a priority, IMNSHO…

​inconsistent naming in ACL’s​

See above. This might be a hill I will be prepared to die on (now I’ve said that, watch me do the same thing elsewhere)!

​prepareNonServerSharedMemory error​

If you see this error in you logs, do weigh in - we are trying to triage how/where it occurs…

​Uploads during rebalance create extra replica​

Not sure why this wasn’t closed by the logical locking of replicas in 4.2.9?

But perhaps I am confusing that with a write-lock MR…

​iadmin modresc rebalance failure​

Fix related to the above, I guess?

​ files have >2 replicas, where one of them is zero length (but not marked dirty)​

This got bumped to 4.2.11

Closed Issues

closed on - 2021-06-21 01:25:42 Codespell report for “iRODS” (on fossies.org)​

I use codespell on this newsletter, its such a useful tool.

closed on - 2021-06-07 23:04:17 release activities for 4.2.9​

Python iRODS Client Activity

Open Issues

​reading large files fails​

This is the report that highlighted the issue that got 4.2.10 planned for ASAP release.

​Long PAM password/token string causes PACKSTRUCT error​

​Force flag not required for data_object.put( ) ​

iRODS client/server convention requires a FORCE_FLAG_KW for overwrites, but the Python API doesnt appear to, and this goes back a few versions.

​Support recursive get and put of collections​

Closed Issues

closed on - 2021-05-28 18:50:49 Keep tests running under client CI and in manual runs​

NFSRods Activity

Open Issues

​Issue copying a file file greater then 55Meg to nfsrods​

TL;DR;

Based on some recent testing, it appears that nfs4j is attempting to write to the target replica in parallel. Parallel writes require the client to provide a replica access token. Using a debugger, I could see that the issue shows up when interacting with the replica access table. I was also able to verify that Jargon can stream large files into iRODS 4.2.9 without producing errors (single threaded). Resolving this issue will likely require adding support for replica access tokens to Jargon.

and also

I’ve learned that not only is this a parallel stream issue, but it is also a caching issue. Disabling caching of nfs4j Stat objects and synchronizing the writes resolved the issue.

Closed Issues

icommands Activity

Nothing this month!

Open Issues

Closed Issues

​