December 2020 Edition of 'The Resource' Newsletter
Hello Reader, here is this month’s iRODS news and developments!
Last issue of the year! Will be returning in January. I usually do these in the middle of the month, but that might not be possible next month, might be earlier or later!
Hey, also, another format change; trying out Markdown. Has it made it more or less readable to you? I've been doing these emails as plain text only, but am considering HTML/plian text combo. Whats your preference?
Hit reply and let me know!
If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere I’ll do my best to let you know. Or just drop me a mail to say ‘Hi’. Always nice to hear from people, particularly in these pandemic times!
I’d love your thoughts and feedback on how this newsletter could be better for you: https://docs.google.com/forms/d/e/1FAIpQLSfODAa7U4ST9U9Tuc6S1PlQhPtoiFybKzXVgtoVnkx7ISe41A/viewform?usp=sf_link
News
TriRods November - iRODS C++ REST API and iRODS Zone Management Tool (ZMT) irods.org/trirods/ https://youtube.com/watch?v=TIfxXKer_fc&list=PL29FhEN41mZNrmk7D7icrqt_fWsi_h6ye
Globus Connector - https://irods.org/2020/11/globus-for-irods-connector-is-released/
Data Management Wageningen Data Competence Centre (WDCC) - iRODs and YODA https://www.wur.nl/en/show-longread/Proper-data-management-makes-researchers-lives-easier-1.htm
Main Repository Activity
In previous editions I had separated out the closed and open issues.
That turned out to be a lot of work, so I’m going to try not doing that this time. Each entry will be tagged if its closed and when though…
Like the new format? Hate it? Let me know!
Move rsDataObjClose finalize logic into APIs - https://github.com/irods/irods/issues/5312 “rsDataObjClose is responsible for the finalize logic of many different APIs (put, repl, copy, phymv, etc.). We want to move this logic back into these APIs so that rsDataObjClose (or some future iteration thereof, e.g. replica_close) does not need to know how to handle various operation types.”
Transient library dependency causes icommands build failure during manpage generation when building against non-package installs in nonstandard locations - https://github.com/irods/irods/issues/5311 Ongoing fine work here making iRODS easier to compile. Thank you!
iput using too much memory? - https://github.com/irods/irods/issues/5309 Investigations into the Consumer using more memory than might be expected when lots of files uploaded.
Add C++ library that simplifies usage of the MsParam API - https://github.com/irods/irods/issues/5308
HPC application access to iRODS data w/o PAM password - https://github.com/irods/irods/issues/5307 - closed on 2020-12-14 19:38:38 Tickets were mentioned here but personally, I think the original requestor missed the fact the PAM auth stores an hashed secret, not the password itself, so would have been suitable…
“istream write” does not set the openmode when using the client-side default resource - https://github.com/irods/irods/issues/5306 - closed on 2020-12-11 22:36:23
migrate to github actions from travis - https://github.com/irods/irods/issues/5302 Poor Travis. I wonder how long GitHub Actions will lastas a Free Tier, though?
Stopping iRODS on the host will kill iRODS servers running in a container - https://github.com/irods/irods/issues/5300 Something to be aware of if you run multiple iRODS servers on one Docker server/K8s pod…
Memory leak in _call_file_modified_for_modification - https://github.com/irods/irods/issues/5299
ifsck should have a way to ignore subdirectories - https://github.com/irods/irods/issues/5296 Its an edge case, yes, but if that edge is lost+found and you have had file system corruption (ahem?), you might want it….
istream should honor client-side irods_default_resource - https://github.com/irods/irods/issues/5294 - closed on 2020-12-11 00:02:18
ninja builds fail for irods/irods master - https://github.com/irods/irods/issues/5301 - closed on 2020-12-12 03:01:45 I’m not familiar with Ninja; https://ninja-build.org/ but having a build system in a build system seems complicated, but I’m sure its for a good reason…
Calling a delay within a foreach of a LIGQ fails when the results are first retrieved and then foreach over each row - https://github.com/irods/irods/issues/5291
already-running delay server doesn’t pick up on new SSL configuration - https://github.com/irods/irods/issues/5290
Test_Resource_Replication_With_Retry fails in CI on bionic - https://github.com/irods/irods/issues/5289 Possibly just an artifact of CI but if you’ve seen similar, let then know?
“iput -rbK " does not honor -K flag - https://github.com/irods/irods/issues/5288 Recursive upload via iput has caused issues. I've also seen this in irsync, so be a little wary.
Clean up ichksum -r output - https://github.com/irods/irods/issues/5285 Finally! I hadn’t realised that the output format was bugging me until this issue. Glad to see its getting reworked.
Add SHA-512 as a hash algorithm option - https://github.com/irods/irods/issues/5282 “This is mainly for the Globus connector but since it has been implemented we decided to go ahead and add it to iRODS.”
“istream write -n# “ does not set the mode - https://github.com/irods/irods/issues/5279 - closed on 2020-12-03 00:27:00
Define ACL meaning - https://github.com/irods/irods/issues/5277 Its not written down anywhere and it diverges from POSIX semantics in a few surprising ways…
inconsistent naming in ACL’s - https://github.com/irods/irods/issues/5276 That’s a paddlin.
setup_irods.py should use systemctl to start irods - https://github.com/irods/irods/issues/5275 …Otherwise systemctl doesnt know its running, with hilarous results. Oh systemd.
Fresh 4.2.7 install server starts but operations fail NO_MICROSERVICE_FOUND_ERR - https://github.com/irods/irods/issues/5274 - closed on 2020-12-02 14:53:15 a consequence of setup_irods templating out core.re at install time, but that file then not being touched in future.
itouch should report itself cleanly to ips via spOption - https://github.com/irods/irods/issues/5272 - closed on 2020-11-24 16:23:04 Have a play with spOption; its can be very helpful for tagging where processes are coming from.
istream should report itself cleanly to ips via spOption - https://github.com/irods/irods/issues/5269 - closed on 2020-11-24 16:22:41
ireg –repl overwrites catalog system metadata for existing replicas - https://github.com/irods/irods/issues/5265
delay server should report itself cleanly to ips via spOption - https://github.com/irods/irods/issues/5264 - closed on 2020-11-24 16:21:16
Overwriting replication with existing checksum results in USER_CHKSUM_MISMATCH - https://github.com/irods/irods/issues/5263 - closed on 2020-11-24 20:14:39
Mark queued delay rules as in-flight - https://github.com/irods/irods/issues/5258
Delayed rule run by a rodsuser fails to update last_exe_time. - https://github.com/irods/irods/issues/5257 - closed on 2020-11-20 21:43:10
Validation of configuration values - https://github.com/irods/irods/issues/5256 A equest to check more values on startup.
ichksum -aK ignores status: down - https://github.com/irods/irods/issues/5255 Which can sometimes be a bonus, and sometimes… not.
ichksum –verify doesn’t verify size of unchecksummed replicas - https://github.com/irods/irods/issues/5252 - closed on 2020-12-12 23:12:07
ichksum -aK on mismatched replicas - https://github.com/irods/irods/issues/5251 - closed on 2020-12-12 23:11:44
Delegate more CMake work to irods-dev package - https://github.com/irods/irods/issues/5250
Allow migration of the delay server - https://github.com/irods/irods/issues/5249
irods-icommands installation fails on Ubuntu20 - https://github.com/irods/irods/issues/5246 Ubunto 20.04 not supported yet, but there are workarounds in the issue for those interested.
Consolidate duplicate symbols in shared libraries - https://github.com/irods/irods/issues/5212 - closed on 2020-11-19 23:26:08 This is helpful if you build it yourself…
close_file after creating a data object in PRC results in intermediate replica - https://github.com/irods/irods/issues/5193 - closed on 2020-12-07 13:29:30
“istream write” ignores –no-trunc when –append is present - https://github.com/irods/irods/issues/5189 - closed on 2020-11-20 00:11:01
istream does not mark sibling replicas stale on update - https://github.com/irods/irods/issues/5187
Delay server updates the log level for the wrong log category - https://github.com/irods/irods/issues/5169 - closed on 2020-11-19 23:26:22
reLog filling with stat error for rei file and stack traces - https://github.com/irods/irods/issues/5153 - closed on 2020-11-20 00:39:34 TL;DR - if you have multiple Providers in a zone, kill the reServers on all but the one you want to run rules.
Create a new metadata interface library - https://github.com/irods/irods/issues/5137 - closed on 2020-12-10 19:00:16
pam_password_max_time does not allow pass_expiry_ts above 1209600 - https://github.com/irods/irods/issues/5096
Fix formatting issues in client_connection doxygen documentation - https://github.com/irods/irods/issues/5092 - closed on 2020-11-24 02:08:16
Deprecation message should not appear when using itrim -n - https://github.com/irods/irods/issues/5045 - closed on 2020-12-15 15:41:59
Use of microservices in dynamic PEPs breaks large file transfers across federation - https://github.com/irods/irods/issues/5017
New replicas which error while creating data on disk are unregistered - https://github.com/irods/irods/issues/4958
iRODS fails to report its status to systemd with supplied sysvinit script - https://github.com/irods/irods/issues/4946
Support for Ubuntu 20.04 - https://github.com/irods/irods/issues/4883 Opened for a while, but new activity - the wonderfully named ‘your-highness’ contributed a workaround to make the icommands install.
iscan core dumps on filesystem permission denied - https://github.com/irods/irods/issues/4613 - closed on 2020-11-19 23:29:04
create new msiAddRErrorMsg microservice - https://github.com/irods/irods/issues/4463 - closed on 2020-11-23 16:55:26
Mounted tar file cacheDir contents not present in the catalog - https://github.com/irods/irods/issues/4457
Delay server should store rei context in catalog - https://github.com/irods/irods/issues/4428 - closed on 2020-11-20 00:12:17
Long syslog messages are being truncated - https://github.com/irods/irods/issues/4365 - closed on 2020-11-24 02:12:03
ichksum should indicate which replica is faulty when run with -a flag - https://github.com/irods/irods/issues/3768
chksumUtil should disallow -R (resource name) and -a (all replicas) from being used together - https://github.com/irods/irods/issues/3540 - closed on 2020-12-15 11:42:08
signalExit/cleanupAndExit should not contain rodsLog calls - https://github.com/irods/irods/issues/3326
feature request: ability to ifsck/ichksum on a down resource as admin - https://github.com/irods/irods/issues/3316
iput: checksum with bulk transfer - https://github.com/irods/irods/issues/3282
move packedReis to db, add delay server boolean to server_config.json - https://github.com/irods/irods/issues/3049 - closed on 2020-11-20 13:25:23
If you think someone else would appreciate this newsletter, they can sign up at https://theresource.metadata.school/
3 Yaks were shaved in the making of this newsletter.