Background:

The design division for an Engineering firm wrote a lengthy analysis of where they see issues with their current business processes.  In the memo, they were not kind to the various software products that have been developed for them (submittal management, proposal management, cad apps).  I was asked me to make sense of what they were saying and I was given a 2″ thick file of all the forms that design needs to fill out during the design process.

I’ve done some research into the tools that we can use for capturing the process that design goes through as well as the data fields that are used in the applications and come up with the following:

1. MetaStorm (www.metastorm.com) – Recently acquired by OpenText this company has a substantial suite of tools for process modeling down to the data layer.  There are several products that they have the first of which is MetaStorm M3.  M3 is their “cloud” based offering which allows you to model a process via your browser.  This costs under $1,000/month.  The second is their MetaStorm Provision (which rolls up their BPM, BPA and Enterprise tools). This suite allows you to capture everything in the enterprise but comes with a price tag around $100k.  The cost for entry into this platform was determined to be  prohibitive for this engagement (it wasn’t in the budget for this year), so an open source solution was sought.

2. Adonis CE  (www.adonis-community.com) – Open Source platform for modeling business processes.  This software does have a paid component to it which allows for detailed Entity Relationship Diagrams but the CE (community edition) tool allows you to create organizations, company maps (general processes), BPM diagrams, document models, risk factors, simulations as well as a host of other tools.

3. Protégé  (protege.stanford.edu)- Open Source platform for developing an Ontology and Taxonomies.  The protégé platform is a robust tool for defining the relationships between entities. The tool allows you to define complex relationships and allows you to prepare pretty pictures of those relationships.

Process:

My initial thought was to use the Protégé tool to model the relationships between the various forms and capture the data fields on those forms.  However, it became clear that the processes that were used to enter the data was part of the problem.  So I switched to using the Adonis CE platform and began modeling the overall design process.

I found during this exercise, that using the document model I can define a simple relationship between the process and what fields needs to be filled out on a form. While not perfect like an ERD diagram I am hoping that it will allow me to show the data field overlap.  The detailed decomposition of the databases will still need to be done using the ERD format.

Using the documents provided by design I was able to create a company map (overview) of the design process and where I have information build out the BP models for the individual processes.  Note the overall organization of information in Adonis is roughly Organization, Company Map, BP Model, IT System, Document Model.

Next Steps:

Continue to refine the model, driving down to the field level for the various applications. Then develop a model of how the fields can be shared (via common lookups) and how data can be pulled from one application to the next.

In writing a chapter for the new release of “Managing Records Retention and Disposal” here in the US, I tried to quantify Records Management from a simple number of documents perspective.

The general consensus is that corporate data doubles every year.  This data is measured in both database records as well as general office type documents.  Given that the majority of Corporate data is document based let’s take the simple example of an organization that starts off with 1,000 documents in 2003.  Assuming a conservative 50% annual growth in documents, that same organization will have  17,00 documents by 2010, 291,000 documents in 2017 and  nearly 5 million documents in 2024.

Granted that this is an overly simplistic example of how document growth can affect an organization, but the point is that if an organization does nothing NOW the problems that they are experiencing will only increase exponentially in the near future.

Now consider the scenario that an organization puts in place records management today and let’s assume that we have applied 7 year retention to all of their documents (excluding vital records of course).   Let’s look at the document growth rate under this scenario.  We start with the same 1,000 documents in 2003, then in 2010 we have just over 16,000 documents, 2017 we have 156,000 documents and in 2024 we have 1.23 million documents.

This is still a sizable repository, and one who’s growth will continue upward, but it becomes much smaller and more manageable than the uncontrolled repository.

The point is that the ability of an organization to deal with their corporate documents is directly proportional to the timing of the start of their records program.  Why do today what you can put off until tomorrow will lead you down a path of greater costs to deal with the issues and an almost insurmountable number of documents to reconcile.  If records metadata is not captured now at the time of creation or during an action on a document, then there is little chance of an organization going back and dealing with 291,000 documents 7 years from now, not to mention 5 million documents in the future.  This pro-active stance will save your organization countless dollars in long term costs.

This also has a direct impact on your selection of an ECM system.  While you may only have 17,000 documents now to deal with, you need to understand the dynamics of your ECM system when it has over a million records.  Speed, performance, management of storage all play a role in how sucessful your organization will be with its ECM system in the long run.  Unless you are the type that enjoys re-deploying an ECM system ever 5 years.

The critical take away from document capture is to use the paper as your disaster recovery source.  Maintain an accurate, up-to-date index of what information is contained in each paper file and box.  Scan the contents of the File / Box and ensure that the index is accessible to your users.  Box retrieval should be a last resort, and the conditions for box retrieval should be reviewed and incorporated back into your processes to see how that can be avoided going forward.  The reason for this is twofold, first box retrieval from storage is costly as the box storage company typically will charge for retrieval and replacement of a box.  Second, the lost productivity that occurs while people are waiting for the box is one of those incalculable soft costs that slow down an organizations ability to respond and react in a timely manner.

Once document images have been captured they should be immediately migrated to a secondary storage system if you are storing them on your network.  If you have chosen cloud computing then you will need to work with your cloud vendor to arrange for secondary (i.e. think cheaper) storage system.  The information that you are scanning is for convenience to your staff and customers and does not require the fastest retrieval speeds.  Given the state of the disk storage community, I would advocate using disk storage for your access and retrieval of scanned images as opposed to a WORM (Write Once Read Many) optical drive technology.

One last note about paper file storage is that whatever index you come up with needs to be consistent between the box storage company and your ecm vendor.  Second, retention rules must be placed on the box information in both locations.  Should the two lists be out of sync having the retention applied in both locations will ensure that at the time of disposition the information can be reconciled.

Capture of document at its creation whether it be from an MS Office based solution or a Multi-Media Marketing campaign is the goal of every Records Manager.  Capture and Classify at creation is the ideal state of operation.  However, few if any ECM vendors (both paid and open) are able to Classify information stored on the “local C” drive.  This represents a vast junkyard of storage for any enterprise.  There is no time soon when content creation will not occur on the “local c” drive and the portability provided by laptops and tablets means that copies of original documents will always remain on the “local c” drive.  This region of the enterprise must therefore be handled through Policies.  Regular sweeps of the C drive for duplicate content or wholesale wiping of hard drives on a routine basis needs to be conducted to ensure that the primary repository is used and has the most current information.

Implementing business process management can help organizations ensure that content is classified after creation and in fact any workflow that runs in an organization Should have an RM classification component to it.  In doing this you will ensure that when disposition time comes, you have a record of all documents and minimize how much will slip through the cracks.

The truth is that both are right

The desktop and server operating system market is the best example that we can look to in order to see how open vs. paid systems can co-exist.  At the time when Linux came to market, prognosticators said that Microsoft, SUN and Novell were dead.  That everyone would be rushing to an open source platform for all of their servers.  The reality is that open source does have a place in the market and only through market forces, consolidation and potentially poor business models has the server landscape significantly changed.

So if we take this model and extrapolate it to the ECM/WCM market place we find that organizations are embracing both open source and paid source systems.  The truth of the matter is that every piece of software has a cost associated with it.  If you pay an industry standard 20% maintenance fee to a paid source vendor, that is a fixed and predicable cost of software.  If you embrace open source ECM, you will still look for support, you will have staff that will spend their time working with the open source community and internally developing new features for your open source system.

The key with any systems that you install is interoperability.  Most ECM paid source vendors are embracing the ISO standards and making their systems more open than just a few years ago.  So long as the vendor continues to embrace these standards then when you have to enforce retention rules and disposition across multiple platforms the function becomes easier. Obviously this whole discussion is predicated on the assumption that you will have multiple ECM /WCM systems in your organization.  If you are just starting down the path o f an ECM system selection then standardizing on one system is easier in the long term. However, most organizations are not in this position and have to rely upon multiple vendors.

Organizations are faced with one major challenge.  They have multiple content management systems and numerous content sources.  Each system has its own terminology and methodology for handling records.  Each system may have a copy of a record that is contained in another system.

This collection of stored corporate knowledge, upon which internal and regulatory controls must be placed, is called the Enterprise Library.  The Enterprise Library, not ECM, WCM or any of those other acronyms, is where the common language of the organization (often referred to as the Ontology) and the common Classification Scheme and the Common Retention resides.

When an organization looks at the collection of content as a whole, the chances of missing something or not disposing of a record properly are reduced.

In the last chapter we looked at one of the international standards OASIS and the CMIS.  CMIS allows for ECM vendors to retain a common repository, while maintaining Records, Security and workflow processes within their proprietary repositories.  CMIS allows for an organization to maintain the multiple Content Systems, which users are used to, keep their existing workflow processes but now leverage the single storage repository that they have sought for all these years.

When you have a collection of ECM systems then interoperability between the systems becomes critical.  The CMIS standard allows for limits of control between the various ECM vendors.  Each vendor is responsible for what is known as Transient Entities.  Transient Entities are entities API’s, administrative entities (such as user profiles), and extended concepts (such as compound or virtual document, work flow and business process, event and subscription) are not included in the specification[1].  This means that each vendor will continue to use their proprietary or open interfaces but access to the underlying content storage is always open.  This repository concept will go a long way towards long term document storage and preservation

Reference MAP of ISO Standards and Implementation below:

ISO 17799

This standard is called the Information Security Program Standards.  Retention of information not only requires storage but it requires security. If you cannot ensure that the information you are storing as a record, whether it be for 1 month or 7 years, is secure and unaltered from the day it was archived then there is not much point to having an archiving or retention program in place.

The standard has multiple components to it including; Conduct Risk Assessments,  Establish a Security Policy,  Compile an Asset Inventory,  Define Accountability,  Address Physical Security,  Document Operating Procedures,  Determine Access Controls,  Coordinate Business Continuity,  Demonstrate Compliance

All of which form a ring of security around the information that you are storing.  Information security has often been referred to as an onion, it has layers.  As with anything, if those layers are not checked on a regular basis (conduct risk assessments) then a security breach can compromise your archives.  Likewise, the ability to Demonstrate Compliance means that you need to have complete audit trails to prove that the information stored has not been modified in the 3, 5, 7 or even 13 years that it has been archived.  In the later case, storing a hash value with the archived document can be a handy tool to show that the original document has not been altered.  A hash value is a mathematical calculation that identifies the contents of the stored document as a unique number.

ISO 15836 – is the Dublin Core Initiative which was started in 1995 and whose goal is to improve discovery standards for information resources.  You can think of the Dublin Core as the guide for the digital tourist. The information stored with each document helps our tourist find and locate the information that they are looking for.  This set of meta data is applicable to both the new employee that is burdened with finding past information related to their position as well as the new employee 20 years from now that needs to understand past decisions so as not to repeat the same work but build on past work.  The Dublin Core metadata element set that deals with Meta Data Standards Consists of 15 Elements (tags);  Description, Publisher, Contributor, Date, Title , Creator, Subject, Type, Format, Identifier, Source, Language, Relation, Coverage, Rights.

ISO 19005-1 discusses the long-term preservation of electronic documents.  This standard is the basis for the PDF/A File Format and provides a framework for the Context and History of a Document.  Standard outlines How a reader should display information, Technical Requirements .File Structure, Graphics, Fonts and Transparency, Annotations, Meta Data and Logical Structure

ISO 15489 The Standard for Records Management. The regulatory environment consists of: Statute and case laws, and regulations, sector-specific, specific to records, archives, access, privacy, evidence, electronic commerce, data protection and information, Mandatory standards of practice, Voluntary codes of best practice

Cmis Standard (OASIS). Provides a standard method for access to a Repository. Standard Repository access provides for multiple ECM systems to operate within an organization.  ISO 14721, The OAIS (Open Archival Information System) standard, which presents models describing digital preservation features and functions

ISO 23081 This standard relates to Information and Documentation for Records management processes.  In other words, Metadata for records.  Records Management Meta Data is used to Identify, Authenticate, Contextualize records, and the people, processes and systems that Create, Manage, Maintain and use them and the policies that govern them

Organizations need to make decisions on which of the metadata requirements outlined in this Technical Specifications they will use.

Metadata supports the business and records management processes by protecting records as evidence and ensuring their accessibility, and usability through time, facilitating the ability to understand records, ensuring the evidential value of records, ensure the authenticity, reliability and integrity of records, managing access, privacy and rights, efficient retrieval, interoperability strategies providing logical links between records and the context of their creation identification of the technological environment  efficient and successful migration of records from one environment or computer platform to another

Records management metadata

Specifications for Metadata are necessary in any or all organizational systems. The choice of what metadata will be included will be dependent on

1. business needs,
2. the regulatory environment, and
3. Risks affecting business operations.

Different views and perspectives on records management metadata are possible and may coexist. These include

1. the business view,
2. the records management view
3. The use view within or outside the records creating business context, Each of these components must be balanced so that when disposition of a record is to occur the Records Manager has all the necessary information available to them to process the disposition.

ISO/TR 18492:2005 This standard Provides practical methodological guidance for the long-term preservation and retrieval of authentic electronic document-based information.  Key Sections of this standard are Section 5, 6 and 7 which discuss the Long-Term Preservation Objectives of your long term archival strategy, Metadata renewal and migration of electronic documents and what the long term preservation strategy for the organization will be.

It should be noted that while the “computer” age is about 50 years old, it’s really only in the last 20 years that the exponential proliferation of information has occurred.  However, we have somewhat of a luxury that during this period many programs that we use today were in existence 20 years ago and those programs can read the old information.  While many program formats have been lost there do exist conversion utilities that will bring many of those formats forward to today’s standards.  So therein lays the rub, the older information formats can be brought forward to today’s standards, but they are no longer in their original “record” format.  So at the point of conversion are they still a record?  I think that the argument can be made that the new copy is no longer a representation of the original record and therefore that record is lost (assuming that the original program no longer exists).

This example is why long term preservation strategies are critical to the organization.  The preservation of corporate records that originate from Office type documents is easy compared to the multimedia / multi format documents that are starting to proliferate in the organization.

ISO 9001 This standard has 4 tiers of required documentation in ISO 9000; Quality Policy Manual, Procedures, Work Instructions and Record Keeping.  The procedures and record keeping sections must contain information regarding records retention and disposition in order to be compliant with the requirements in ISO 9000.  This requires that procedures be kept for the organizations major processes as well as records are kept of critical operations. The important take away from this ISO standard is the documentation and procedure for how records are disposed of.  A record that has existed in an organization for 7 or 13 plus years is not always going to have the luxury of individuals with institutional knowledge to determine  if the record is still of value.

In relation to long term preservation of records and record disposition this standard is important not only for marking records, but for determining if it is the correct time to dispose of records.  The disposition process must be rigorously enforced so that proper sign-offs are carried out and that disposition is carried out according to a standard corporate process.

Records Management Systems, which were once stand-alone platforms that mostly handled box management, have now been merged into larger organizations and now make up a component of a larger information management offering.

DM or DMS

Document Management Systems encompasses the functions of a repository, check-in and check-out, version control and access security.

EDRM or EDRMS

Electronic Document and Records Management Systems encompass both document management functions and RM functions, such as classification, retention, and disposition.

ECM

Electronic Content Management is the suite that includes the full life cycle of a document from creation to disposition.  It generally includes the ability to archive records based on criteria to alternate media.  An alternate term used for this class of software is RDIMS (Record and Document Information Management System).

WCM

Web Content Management is the class of software that focuses on the creation and management of content on intranets and public facing web sites.  This class of software does not necessarily have a records management component to it, although it does have the ability to capture web sites so that they can be archived by a records management system.

Record System Terminology:

• Physical Record – This is just as it sounds: the parchment, paper, velum, Mylar, or other physical representation of a set of thoughts that constitute the official transmittal to another party.  This “Record” can be captured as a microfilm, microfiche, or electronic (scanned) image of the original.
• Electronic Record – The official transmittal of business information in any digital media (office formats, scanned image, database entry).
• Work-In-Progress – Any set of documents (often referred to as Drafts outside the RM community),  consisting of mark-ups, unapproved comments, and/or documents that may express the initial thoughts of an organization.  It should be noted that while Works In Progress are not necessarily official records, they must be subject to retention as they are discoverable.
• E-Discovery – A process by which sets of keywords are used to search electronic information within an organization for the purposes of a legal action such as a law suit.
• Semantic Search – A search that attempts to find information in a document based on the context of the search term(s).  Unlike a Keyword Search, which  returns results based on how frequently the word(s) are used in a document, the Semantic Search looks for meaning, and thus may return results even when the exact word(s) searched is not even present in a document.

There are also a number of Acronyms commonly used in the Technology community. Below is a list of the most important Technology Acronyms:

1. WCM – Web Content Management
2. DM – Document Management
3. RM – Records Management
4. ECM – Electronic Content Management
5. EDMS – Electronic Document Management System
6. EDRMS – Electronic Document Records Management System

When using an ECM package that has a folder structure as a part of its core method of storing documents (ie #Opentext or #SharePoint) there are many ways to implement this folder structure but the challenge is to balance out the needs of security, metadata, records management, and ease of access.

So the question becomes what is the best structure out there?

While there are a large number of combinations I see it breaking down as follows

1. Department centric
2. Records Management centric
3. Process centric

Department centric

This is where the users see information based on the context of their organizational structure. Sometimes referred to as an ontology the main goal of this view is to give quick access using the language that the user is familiar with.

Records Management Centric

This is where the users see information based on how the organization has developed is classification scheme and retention schedule.  This structure is usually the most complex and comprehensive approach to an organizations information.  Similar to walking into a Library and seeing the categories of books.  This view will generally have many layers (think folders) until you get to the documents.

Process centric

The most comprehensive view of an organizations information.  Information is organized based on the business process, the information moves from location to location based on its status.

Each of these views have been successfully deployed in many organizations.  What’s right for your organization and culture will depend on how well you can reduce the learning curve and increase user adoption.

Over the next several weeks I will be posting drafts of the sections that I will be including in Chapter 4 with the hopes of soliciting feedback from the community at large.

As it stands now the outline for Chapter 4 is as follows:

Chapter 4 – Technology and Records Retention

1.       Introduction

• Technology solutions – not the same as paper
• Business Process – the same as paper
• Strategic Importance of Electronic Content Control vs Unstructured Data

2.       Record System Terminology

• List of Record System Terminology
• Technology Acronyms used

3.       Types of RM Systems

• Pure RM
• Box Management
• EDRM
• ECM

4.       ISO Standards and their Impact on Technology (the way things should be done)

• Overview
• ISO 17799
• CMIS Standard (OASIS)
• ISO 15489
• ISO 15836
• ISO 19005-1
• ISO 14721
• ISO Standards that will be referenced :
  • ISO 23081
  • ISO/TR 18492:2005
  • ISO 9001
  • IEC 82045

5.       Enterprise library (they way things are done)

• Definition – The collection of stored corporate knowledge, upon which internal and regulatory controls are placed.
• Collection of ECM Systems
• Interoperability between ECM
• Limits of control between ECM systems

6.       Open Source vs. Paid ECM systems

• Overview of the Features Present In Open Source Systems
• PAID (need a better name) ECM Systems
• [Optional] Gartner Review of Industry

7.       Document Capture

• Paper File
• Office Document

8.       Workflow

• Business Process Modeling
• Mapping business process to Records Classification / Schedule

9.       Email Capture

• User Initiated
• Automated
• Hybrid Approach

10.   Remote User Support

• Remote Cache Server

11.   Cloud Computing

• History of Computing (brief)
• What is the Cloud
• Regulatory Impacts on the Cloud
• General Issues faced by Cloud Computing

12.   Social Media / WEB 2.0

• Web Sites
• Instant Messenger and E-Rooms
• Facebook and Similar Presences
• Twitter

13.   Long Term Preservation

• PDF/A Format

Feedback and Suggestions are More than Welcome!