When I first joined The Journal of Bone & Joint Surgery (JBJS) in 2019, I started on the Production team. It became clear very quickly that this was not a good fit. I was used to working with automated content systems, and I was dismayed to find that JBJS handled everything manually. I moved over to the IT team, where I was eager to apply my experience coming from a large publisher, and build an automated content delivery system for JBJS.
In the new content delivery system, JBJS's publisher, Wolters Kluwer, drops publication-ready files into a "hot folder" which organizes them, checks them against a list of business rules, and delivers them to JBJS's content repository, RSuite CMS, which then presents a validation report and an accept/reject decision point to the RSuite admin. Once accepted, the files are organized into RSuite's folder structure, and delivered to Cog9, the admin portal of JBJS.org. From there, the system admin may review the articles on the Staging site, and publish them.
I served as the project lead, defining the problems to be solved, and delineated the project requirements. I worked with the team from Orbis (the company that built RSuite) to develop and test the the content delivery system. I also worked with JBJS's web development vendor, Area9, on enhancements to the process of content receipt, and served as point of contact between all parties across 4 companies and at least 5 countries.
1 advisor, 1 product manager (me), project manager and developers from Orbis, developers from Area9
The UI (user interface) of the system was limited by RSuite's product design. For example, we could not move buttons from one side of the screen to the other.
October 2019 – July 2020
My former job at SAGE Publishing (which also uses RSuite) meant that I already had 6 years of experience working with an automated content management system in RSuite.
During the few months that I was part of the Production team, I became very well versed in the publishing workflow. When I moved departments, I agreed to take RSuite management with me, so that I would not be leaving Production short-staffed. So in this case, the user I was designing for was myself!
The publisher, Wolters Kluwer (WK), has their vendor place publication-ready files in an sFTP folder and (inconsistently) alert the Production team.
Production downloads the files, and organizes them into a "content library" on SharePoint.
The RSuite admin opens each XML file and checks known problems areas, requests corrections if needed, uploads all files into RSuite, and notes metadata and RSuite ID on a tracker spreadsheet.
On the scheduled publication day, the RSuite admin copy-pastes each RSuite ID from the spreadsheet into the JBJS.org ingestion tool, and runs the process.
100% time-consuming manual workflow
I estimated this process takes about 10-12 hours a week.
The ingestion tool alone takes ~45 minutes on each run, and a person must be present to click through the buttons; if the computer sleeps, the cycle fails and requires a restart.
High opportunity for human error
Relies on human eyeballs checking XML files for inconsistencies. Things will be missed.
The "content library" is 245 GB of data as of March 23, 2020. This duplicates content that already lives in RSuite.
Articles cannot be checked on Staging ahead of publication. Articles are published, and necessary corrections are noted after the fact, and sometimes not implemented for months (or in some cases, years!).
Articles cannot be scheduled for publication in advance. WK is supposed to avoid holidays and weekends, but WK and JBJS do not have the same holiday schedule, so sometimes I had to log in and run the ingestion tool on a holiday.
"Hot folder" ingestion – instruct WK to place articles into a designated folder that is configured to, at intervals, deliver any files it finds within in to RSuite
Automated XML checking – build a schema of business rules to run verification of each XML file
Automatically deliver articles from RSuite to JBJS.org Staging area
Implement a dashboard in the JBJS.org admin controls to view and publish staged articles, including scheduling publication in advance
I asked RSuite's company, Orbis, and JBJS's developers, Area9, for broad estimates of the work required, and wrote up some slides. I presented them first to my manager, and then to senior management. I emphasized the amount of time to be saved, and that strengthening our publishing infrastructure would be an important step for the future development of JBJS.org. The project was approved, and SOWs (statements of work) were signed. And so we began!
Orbis gave me access to an Rsuite testing environment, and invited me to their JIRA board. We met weekly so they could keep me apprised of their progress, and ask me questions that needed more discussion than JIRA comments could provide.
I wrote up a list of business rules that every XML file should be checked against. Orbis built the Schematron of business rules. Their developer also very helpfully walked me through what he was doing in our weekly meetings, so that I understood the logic. I did a great deal of testing, and even got to the point that I was able to write new rules myself!
Each new article is validated automatically, and we added a new menu option so that validation can also be initiated on existing content as needed. Every validation creates a report, which is attached to the article in RSuite, and also delivered to the RSuite admin by email.
The workflows were mostly built with Orbis, but now there was the need for collaboration with Area9 on delivering content to the JBJS.org admin site, Cog9. This was probably the most challenging part, because it involved people across at least 5 different countries and time zones.
Once content is loaded into RSuite and run through validation, RSuite creates a task for a the JBJS admin, called the “gatekeeper task.” The gatekeeper chooses to “approve” or “reject” the article to complete this task, making a decision based on the errors/warnings reported in the validation report. Approve will initiate the “File and Deliver” workflow. Reject will initiate the “Rejection” workflow.
This workflow moves approved files from the staging area in RSuite to the appropriate hierarchically-structured folder (based on the journal, volume, and issue metadata). If no container exists (e.g., in the case of a new issue or volume), a new folder is created. An article with a Digital Object Identifier (DOI) that already exists in RSuite will load as a new version with the same DOI.
Articles with unpublishable errors are rejected during the "Gatekeeper" workflow. They are moved to a temporary holding folder and cleared periodically.
When content is placed within the sFTP "hot folder," it ingests into RSuite and begins the validation process. WK's vendor delivers the content, so we gave them a new sFTP address and login details.
We initially designed the folder to process 1 article at a time, but a complication arose when WK's vendor refused to change their system of delivering articles in batches. So, we ended up creating a second hot folder. The first unzips the batch folder, and organizes all the files within into individual articles, and then moves them to the second hot folder, which validates and ingests them into RSuite. This required an additional SOW, since we had not planned for a second hot folder.
Also, we discovered that some larger files triggered the delivery workflow before the sFTP file transfer was complete, resulting in an error. To fix this, we implemented a delay where files would not trigger the workflow until 10 minutes after the transfer finished.
Every morning, RSuite delivers a report detailing the previous day's content received, including the publication date from the XML. This is useful for members of the Editorial team who track time to publication data.
At this point in the process, Orbis's part was done. The final phase was completed with Area9.
When Cog9 receives RSuite IDs via the API, it now ingests the articles to its staging database, where they can be checked prior to publication. We implemented a Staging dashboard, which displays relevant information regarding each article, and allows publication to be scheduled in advance.
The final phase went live in July of 2020. I've used the system for ~4 years now, and it is a massive improvement over the old manual system. It does a better job ensuring the quality of the content, and in far less time. I spend ~2 hours a week managing content publication, instead of 12 hours, and Production never had to hire another person to replace me.
All process documentation files are saved in RSuite, and I wrote up careful step-by-step instructions for how to use the system. I also ran a training session by Zoom. I was still the primary user, but having trained backups ensured that when I was out of the office, someone else would be able to step in seamlessly, and without having to take on a massive amount of extra work.
“Isabella expressed a strong interest in implementing automation with RSuite to improve the flow of digital content from WK to JBJS, specifically JBJS.org in the short term. Since the inception of this project last fall, Isabella has presented the plan and project timeline to senior management and has successfully and independently led the multi‐company project team to a successful launch on time and as close to budget as possible (she did encounter some late unexpected technical obstructions from WK). The RSuite automation project will be the foundation for expanding digital production processing automation throughout JBJS production and editorial processes, which will reduce costs, cycle times and improve process/product quality." – my manager
This was my first technical project that I directed solo, and I'm still incredibly proud of it. It was a fantastic learning experience: designing and testing the system, and herding all the cats involved. I also learned how to code Schematron rules. As a former English major, I never would have thought I'd be able to do that, and yet, once I understood the logic, it wasn't hard. It was even quite fun!
In the beginning, I was faced with the attitude of "well, we've always done it this way." In order to foment change, I accepted extra work onto myself and took ownership of the system. And then, once it was my system, I was free to change it as I liked.
In retrospect, I should have run the plan by our publisher first, rather than just assuming their vendor would change their processes at our request. Though considering their habit of giving us different answers when a question is posed again, it might not have helped anyway.
Currently, JBJS is tied to WK, which also publishes our content on their own site. However, having two independent sites is confusing to our readers, and if and when WK's site is ever discontinued, a strong content management system is important groundwork for the future. We could also look into using RSuite to store and track files undergoing copyediting and production.