2024 APRIL 30

In this month’s issue:

1. MPC Documentation Upgrades

2. “New” Documentation

3. PostgreSQL replicated table

4. MPC Users Group Meeting

1. MPC Documentation Upgrades

The MPC continues to work on updating the documentation structure of our website, moving all our existing documentation to Django, with the goal of removing legacy dependencies from Fortran and Ruby-On-Rails code and making the documentation more easily accessible and hopefully clear. The new development will remain under NGINX, which has proven to work extremely well for us.

Some major upgrades in our code have been needed in order for us to be able to code and test everything locally, without disrupting the current functionality of the website, which we know is crucial for our users.

This Newsletter will be mostly focused on these new upgrades.

2. “New” Documentation

A new item on the navigation bar

A “new” documentation written using Django code was already available from our website, but it was hidden in the main menu under OBSERVER → Documentation → Other. The page contained all the information about some of the newly developed services. We agree that it is not easy to find and we have decided to move the link to the main navigation bar under the word “DOCUMENTATION”.

The Documentation page

The DOCUMENTATION page contains the MPC’s documentation for some of the most recent developments, including the MPC’s APIs. In addition to that, we have recently migrated the postgres-related documentation to this page, as well as the ADES documentation with the addition of some useful lists.

MPC’s PostgreSQL replicated table

In our March 2023 and May 2023 Newsletters we gave a general overview of how observations are stored and published at the MPC and how they are replicated to SBN. The Section PostgreSQL replicated tables will also try to clarify some of the main issues that have been recently reported and addressed by the MPC. Some documentation was available from our website at the following link:

https://docs.minorplanetcenter.net/mpc-ops-docs/data-and-services/replicated-tables-intro/.

Following your comments and suggestions, we have created a new documentation page for the schema that also includes useful links and we hope it will provide more articulate information. The new page for the PostgreSQL schema is now available here.

It contains a brief description of all the PostgreSQL tables that are replicated to SBN, with a link to a dedicated table for each page.

The pages for every table contain a description of the table, useful tips on how to use the data, and information on work in progress or data structures. We also provide a list of any known issues in cases where we know that the table requires updates. In addition, every field description contains links to other useful documentation pages that should help explain its content. For example, please check our new page for the obs_sbn table.

What do you need to do?

• If you had bookmarked our previous page, please remove the bookmark and add this new page to your bookmarks.

• If you have any questions, comments or suggestions, please contact us.

What are we working on?

• Only the schema page has been moved to Django, the page containing examples and instructions is still in the old format. We will migrate the page and update the instructions in the next month.

• We will deprecate the old schema page in the next 30 days. The link has already been added to the Deprecated webpages page.

• If new PostgreSQL tables are going to be replicated to SBN, the documentation will only appear on this new documentation page.

ADES documentation

The Astrometry Data Exchange Standard (ADES) is a data format for astrometry, providing many new features over the 80-column format (for a full description of the ADES format, please see our August 2023, November 2023 and February 2024 Newsletters).

As part of this work, we have migrated the MPC’s ADES documentation over to Django, and updated numerous sections. The ADES documentation is now accessible from the main Documentation page, with additional links to the ADES GitHub repository. In addition, the list of valid ADES values has been updated.

Program codes

The Minor Planet Center is responsible for assigning program codes to identify different observers observing from the same telescope. The main difference between the MPC1992 80-column format and the ADES format is that the prog field in ADES (and in the obs_sbn table) is a representation in base62 of the progressive number that has been assigned to each program code.

The MPC was already maintaining a list of program codes, but that list did not include the program code in base62, making it rather complicated for our users to compare the program code from the 80-column format to the prog field in ADES or in the PostgreSQL obs_sbn table. To rectify this, we have added a new page to the documentation, containing a table with the program code single character as it appears in the 80-column format, the corresponding progressive number as assigned by the MPC, the base62 conversion of the progressive number and the contact name(s). We use the additional_coinvestigators field to list all the contact names associated with a given program code.

3. PostgreSQL replicated tables

Tables summary and status

We provide here a summary of the replicated tables that have been replicated to SBN. We note that the status of completeness and reliability of the various tables is not the same, as the population of some of these tables will occur in the future (i.e. some of the current tables act as placeholders). The documentation for every single table should now contain all the necessary information to understand and interpret them: please let us know if you feel we can provide further clarification and or documentation.

• All the NEOCP related tables.

  • All the tables are good and ready to be used. The MPC completely relies on those tables for our internal NEOCP processing.

• The current and numbered identifications tables.

  • The tables are complete, up-to-date and ready to be used. The MPC completely relies on the identification tables. Moreover, the designation API, the observations API and the MPC Explorer are currently using those tables as their primary source of data.

• The primary object table.

  • The table is complete. Feel free to use it.

• All the obs alterations table.

  • All these tables are still a work in progress. Some of them may contain some data, but others (e.g. obs_alterations_redesignations) only serves as a placeholder for the time being.

• The observations (obs_sbn) table.

  • The obs_sbn table is a replica of the MPC internal PostgreSQL table. As such, it contains exactly the same information that we store in our internal PostgreSQL table, with the only caveat that the replicated observations are the ones that have already been published or that are in the ITF. You are not able to see observations that we have deprecated or that are waiting to be processed.

  • The obs_sbn table is ready to use. The MPC fully relies on its internal obs table as primary source of data. There are some internal inconsistencies, but they do not affect the overall quality of the data (see our March 2024 Newsletter).

  • The Observations API and the MPC Explorer get their data from our internal observations table. The goal of the MPC Explorer is to create ADES files that would pass the ADES validation. As such, some of the inconsistencies are cleaned up by the API before the file creation. We are actively working on removing all the remaining inconsistencies. In the meantime, we have added to the Explorer a description of the Known issue and their status (as mentioned in the March 2024 Newsletter).

• The orbits (mpc_orbits) table.

  • The mpc_orbits table is up-to-date with the most recent data, but we haven’t been able to run a full comparison with the flat files, as we did for the observations table.

  • You can still consider this table as a work_in_progress.

  • We are aware that some fields are still not populated for the majority of the objects.

  • We also know that having a reliable orbit table is important and working on the mpc_orbits table is next in line in our list of priorities.

4. MPC Users Group Meeting

The MPC Users' Group is composed of representatives from the major NASA-funded NEO surveys, the NEO follow-up community, the dynamics community, and the simulations community. The goal of the MUG is to provide feedback to the MPC about its current status and future developments. The MUG members and the MPC staff usually meet twice a year. The next meeting will take place on May 21-22 at the Center for Astrophysics, in Cambridge (MA). The MPC welcomes any feedback from the community (including positive ones!) and we encourage users to share their thoughts before May 16th with the MUG representative Rob Weryk (rweryk@uwo.ca).