Hello Reader, here is this month’s iRODS news and developments!

If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere I’ll do my best to let you know. Or just drop me a mail to say ‘Hi’. Always nice to hear from people, particularly in these pandemic times!

I’d love your thoughts and feedback on how this newsletter could be better for you.

News

User Group Meeting

iRODS User Group Meeting 2021

42 talks (slides and videos) now available

iRODS S3 Resource Plugin 4.2.9.1 is released

Fixed a redirection bug in the new streaming behavior.

​4.2.9.1​

Main Repository Activity

Open Issues

​PAM authentication failed in icommands during long connection​

As mentioned in the comments, probably the known issue of unable to refresh the PAM token timeout (#5096, #4198, #3742).

​irods-rule-engine-plugin-update-collection-mtime in repo depends on 4.2.8​

This caught me out on the irods_demo repo PR I have been working on (ahem, since the UGM… ), as upgrades appear to do the right thing and uninstall the package. However new installs from the irods repo will fail if this package is included, such as in the aforementioned repo. My MR has pinned the version to 4.2.8, but the right thing is probably a different MR to remove the package as by default it deployes the latest release - currently 4.2.9.

​Packaging: Conflicting packages should be marked as such so that package managers do the right thing​

This was on Ubuntu, I am not sure Centos et al were checked.

irods-database-plugin-oracle and irods-database-plugin-postgres conflict, so they should be marked as conflicts in the packages.

Some discussion - how do YOU think this should be resolved?

need to confirm the mysql package does the same - they should each three declare conflicts with the other two.

or

A perhaps heretical opinion - why should they conflict at all? Each one could use its own uniquely named SQL file etc al.

If you have an opinon, weigh in in the comments!

​Object locking/state issues when iput/irm clients contend heavily for one file​

TL;DR - when doing lots of simultaneous writes to the same file, only iput -f will leave the catalogue in a usable state for the file on 4.2.9.

​iquest parser: this should error, but returned nil​

Good tip in this about the use of spLogSql=1 to get further debug logs (its in the docs, as I’m sure you know…).

Probably(?) won’t be fixed for 4.2.10, as it may be addressed by 3902, which is plannd for 4.3.0…

​irods breaks mysql 8 GTID replication​

Worth reading if you use MySQL as a back end with some kind of High Availability design. I wonder if this would be the case with other HA database implementations? It would seem likely?

​ iquest cannot match names containing apostrophes via = operator if it’s not the last condition in the where clause​

​Recursive irepl silences failures​

Worth noting if you use irepl - do put your opinion on how this should behave in the Issue;

When running irepl with the recursive flag on a collection, any errors encountered are kept quiet. This has been the case as early as 4.2.8, but possibly earlier. Is this the expected behavior? If not, should the user expect a message for every encountered error (up to the size limit of the rError stack)?

​irods-grid status produces errors in rodsServerLog (race condition?)​

Race condition in irods-grid - worth reading if you use it to programmatically check for processes.

​ils, imcoll: accidental string concatenation of string literals in array initialization​

​lsb_release, Python’s platform, python-distro, and /etc/os-release​

Reducing the hidden complexity / requirements is always nice to see (and hard to do!)

​Add full utf8 support to mysql​

I am slightly boggled it didn’t have this already? Just me? I seem to recall adding support to Oracle in 3.3.1 was tricky, though.

​How to validate a database migration?​

If you use PostgreSQL worth checking out the Utrecht authored tool mentioned here. If you don’t… Why not write one for your chosen database?

​release activities for 4.2.10​

Fingers crossed!

​prepareNonServerSharedMemory error​

Still under investigation and discussion. I comfess I still dont understand Shared Memory allocation on Linux…

​replace GenQuery ad-hoc parsing code with flex/bison parser​

Mentioned above, targeted for 4.3.0

​set socket options to avoid timeouts​

Would be great if iRODS could do this, rather than requiring the host OS to be configured.

Closed Issues

closed on - 2021-07-15 20:11:22 irepl -R resc_with_a_stale_replica should update the stale replica​

Reported on 4.2.7, fixed in 4.2.9…

closed on - 2021-07-06 15:53:37 Replacement for NEWLY_CREATED_COPY #define removed in 4.2.9​

If you write your own C clients, this is worth knowing to support >= 4.2.9.

closed on - 2021-07-14 23:03:58 failed to start irods server after compiled ​

This makes me sad that externals doesn’t appear to be versioned to match the iRODs version - so building earlier versions is hard / not garunteed to be reproducible. I thought an issue had been raise for this, but I can’t find one now…

closed on - 2021-07-22 14:09:21 A grab bag of questions and previews of DFC noted client enhancements​

Where DFC is DataNet Federation Consortium I think…

Python iRODS Client Activity

Open Issues

Closed Issues

closed on - 2021-07-07 20:37:32 reading large files fails​

Where large >= 2G - this is actually the 4.2.9 issue, the prompted the immediate work to release 4.2.10.

closed on - 2021-05-14 20:15:16 Atomic operation on metadata errors with symbols in the value​

This issue will have to be addressed in the upcoming iRODS 4.2.9 release with the resolution of irods/irods#5520 and a corresponding release of the Python iRods Client (PRC) . Meanwhile, we recommend following the traditional approach to metadata in PRC when adding tags that could contain a “<”, “>” or “&”.

NFSRods Activity

Open Issues

​Does not run without /dev/stdin binded to docker​

I’m no Docker Guru, so I appreciated the further explanation;

Commit 318663d makes running the docker image without the -i flag crash immediately, as /dev/stdin is absent. This makes a container workflow completely useless.

​Issue copying a file file greater then 55Meg to nfsrods​

Some notes from the investigation that might help get your heads around the issue and its potential fixes;

Based on some recent testing, it appears that nfs4j is attempting to write to the target replica in parallel. Parallel writes require the client to provide a replica access token.

and

I’ve learned that not only is this a parallel stream issue, but it is also a caching issue. Disabling caching of nfs4j Stat objects and synchronizing the writes resolved the issue.

(The comment then goes on to detail the required changes).

Closed Issues

icommands Activity

Open Issues

​better verbosity for writes​

Do you have an opinion on how the verbosity flag should behave on the icommands? If so this is the Issue for you! Weight in on the comments with your use cases so the community’s requirementsn can be captured…

Closed Issues

If you think someone else would appreciate this newsletter, they can sign up at https://theresource.metadata.school/​

Three Yaks were shaved in the making of this newsletter.

Unsubscribe