Pogo PossumWe have met the enemy and he is us.” – Pogo (by way of Walt Kelly)

To make the Automated Information Lifecycle Management model work – and it must work – some serious challenges must be overcome by the three groups most affected by it: Records and Information Management professionals, Enterprise Content and Records Management product vendors, and records and information management services customers.

In this article, I’ll discuss the challenges those of us in the Records Management profession must resolve to successfully implement the Automated Information Lifecycle Management model. In subsequent articles, I will address the issues which the AILM model create for records and information management services customers and ECRM product vendors.

Records Management – really the effective management of recorded information through its lifecycle – has been a noble profession for thousands of years. For the vast majority of that time, we were tasked with maintaining, safeguarding and disposing of eye-readable information recorded on some form of physical media – whether carved in stone, painted on papyrus or printed on paper. But those few millennia are behind us now. Information recorded on paper is quickly becoming a novelty. In fact, for many organizations, it is already a liability.

Unfortunately, like any other profession that has existed for centuries, the methods we employed in the past have become deeply rooted in our culture. We have become so good at managing paper records that it is part of our collective DNA.

During the mid-90s, when the volumes of electronically stored information were first beginning to dominate physical records, and it became clear that this would be an irreversible trend, records managers were some of the first to address the changes this would require. But though our motives were pure, our deep seated culture biased toward physical records management wanted us to manage this electronically stored content using the same procedures we used for so long with paper-based records.

Thus began a twenty-year period of explosive record growth, cost prohibitive discovery requirements, lost institutional knowledge and a generation of information workers over-burdened by poorly defined and ineffective records management requirements.

So this is the challenge we now face. If records management is going to succeed in the 21st Century, those of us in the profession must be willing to remove ourselves from a physical records mindset and fully embrace the notion that technology provides us unique ways to manage the lifecycle of electronically stored information that are exponentially better than how the same information was managed in a paper format.

In practical terms, this means:

  • We must accept that the massive volumes of information created with modern technology mean that no records management solution will ever be perfect and a well-documented solution, planned and maintained in good faith, is good enough.
  • As more and more information traditionally created in an unstructured format moves into databases, the lifecycle requirements of structured information must take on an increasingly greater records management focus.
  • We must massively simplify our records management processes. Big Bucket retention schedules must be implemented enterprise-wide. Burdensome legacy file plans must be substantially streamlined. Multi-stage retention requirements, originally necessary for managing paper records, must be dramatically reduced or even eliminated. And content analytic tools must be leveraged to provide auto-tagging and auto-classification at every opportunity.
  • And we must make every effort possible to eliminate manual processes from the information lifecycle. End users must no longer be required to ‘declare’ and classify a record. Requirements for the manual approval of routine records management procedures, such as disposition and transfer, must be reduced to only a small number of the most critical records series. And the information lifecycle requirements of transactional material produced by routine business activities must be baked into the process itself, becoming completely transparent to everyone but those few people focused exclusively on records management.

The stakes could not be higher. Over-retention and over-preservation of information is the IT challenge of our times. It is the root cause of every other information management problem our organizations face. And only trained, experienced professional records managers using the latest technology to automate information lifecycle management can solve it. But first we have to free ourselves of some very old, very powerful cultural bias.

SharePoint is a ToolImagine you’re alone at night, lost in the woods. It’s getting dark and threatening to rain. You have the equipment you need to build a good fire, but what you really need is shelter. Fortunately, you brought an ax with you. That’s the good news. The bad news is, the ax is just a tool. The ax, by itself, is not going to solve your problem. How you use the ax to build a shelter (the actual solution to your problem), is entirely up to you. If you use the ax to chop down a few saplings to lay across your body, you haven’t used it well and you may freeze to death as a result. However, if you use the ax to chop down several young trees and craft a sturdy lean-to, you may have saved your own life.

And so it is with SharePoint. SharePoint is simply a tool. No more and no less. It is how you use SharePoint (and all the other tools in your arsenal) that will ultimately determine the success or failure of whatever solution you create.

Sadly, SharePoint has developed a bad reputation within a lot of organizations – especially as a tool to support records and information lifecycle management solutions. Some of that reputation is deserved, but not most of it.   The majority of the bad word-of-mouth that SharePoint gets is the result of end users associating SharePoint (the tool) with poorly planned and executed solutions created using SharePoint as a platform.

Here’s an easy and extremely inexpensive trick that I can guarantee will increase the adoption of your SharePoint-based solution: stop calling it ‘SharePoint’. Give the solution a name. The Integrated Content Environment (ICE). Or the ‘Zone’. Or the ‘Info Hub’. Or whatever. Just don’t refer to it as ‘SharePoint’.

Consistently refer to it by the name you’ve given it – and encourage your peers, your end users and senior leadership to do the same – and all your stakeholders will begin to see your solution as an integral part of the success of your business rather than the flawed information lifecycle management product they may perceive it to be.

E FormsAs I’ve discussed in this blog (and elsewhere) on a number of occasions, as organizations begin to move their routine, high-volume unstructured content (e.g. invoices, bills of sale, etc., etc.) into structured items in databases, forms management will become an increasingly important aspect of records and information management policy. (After all, just because the information has moved from an unstructured format to a structured format doesn’t mean the requirements for managing the lifecycle of the information have changed in any way.)

That’s why it was important to us all when Microsoft announced around this time last year that they had officially decided to ‘deprecate’ (read ‘kill’) InfoPath and declared that new forms engine technology would be released sometime in the foreseeable future.

And then at last year’s SharePoint Conference, Microsoft announced they were actually working on several different options for replacing InfoPath:

  • Excel Surveys – a solution which, not surprisingly, will leverage existing Excel features
  • Word Form Documents – which will use Word to create structured forms
  • App Forms – which will use Access databases to generate forms

And finally, the option I thought had the most promise and slickest looking user interface,

  • Forms for SharePoint Lists – which will allow for the creation of configurable forms based on…wait for it…wait for it…SharePoint lists

Well, shows you what I know. With very little fanfare, Microsoft recently updated their Office 365 Roadmap to include the following announcement:

Click to Enlarge

This is not an insignificant announcement. Forms management will become a critical feature in SharePoint given the accelerated importance of mobility in the workplace. Microsoft must provide a competitive forms product offering in the SharePoint stack if they expect to compete with the newest set of competitors – most of whom were designed with mobile functionality in mind from the very start.

Let’s hope one of the other three potential InfoPath replacements fares much better…

 

ElvisThere has been a small, but vocal group of analysts over the last year or two who have been publically speculating that Microsoft, always trying to drag the general public, kicking and screaming, into the cloud, would not release another new on-premises version of SharePoint. Apparently, those analysts were wrong. At least partially.

Microsoft will, indeed, release a new version of on-premises SharePoint according to this recent post on the company’s Office blog. (I say the analysts were partially wrong because I am inclined to agree that Microsoft would not release a new on-premises version without substantial prodding by members of its current customer based – many of whom are still reluctant to move into a cloud-based solution for one reason or another.)

The new version will be cleverly titled ‘SharePoint 2016’ and is scheduled for release sometime toward the end of this year. It is not entirely clear what, if any, new records management functionality will be included in this release.  Though Microsoft will continue with its tradition of the last few years of taking features developed in the cloud first and moving them into the on-premises version, so I wouldn’t expect to see any big surprises.

Microsoft promises to reveal more details on SharePoint 2016’s new features and roadmap at their upcoming Ignite conference in Chicago during the first week in May. We will likely have to wait until then to hear more.

[On a personal note, this article marks the fifth time I’ve managed to include a picture of Elvis in one of my posts. I think that’s a lot, given that this is a blog about SharePoint records and information management. And I just want to say, ‘You’re welcome. I am here to serve.’]

CliffMalcolm Gladwell is arguably Canada’s greatest living gift to Western Civilization (outside of Bob and Doug McKenzie, who are hilarious).  In his seminal book of the same name, Gladwell defines a ‘tipping point’ as “the moment of critical mass, the threshold, the boiling point”.  And unless you’ve been living under a rock somewhere – or perhaps been chained up in a physical records repository blissfully managing paper documents from the late 1970’s – you could not help but notice that the current state of records management has reached a classic Gladwellian tipping point.

The decades-long mismanagement of electronically stored information, the billions of dollars wasted on failed records management solutions and the explosive growth of newly created information have all brought us to the brink of disaster.   We are all standing on the edge of the same rocky cliff and we’re staring into a pit full of costly, dangerous and unreliable information.

The current rate of over-retention of information is simply not sustainable. This is true from a cost perspective, a legal perspective, a business efficiency perspective, or any other way you may look at it. And you can argue that storage is cheap and getting cheaper all the time. I’m not going to disagree with you on that. But the fact is the basic math is very clear here: storage costs are not declining at a rate anywhere close to the exponential rate that new information is growing.

And honestly, when we’ve reached a point where companies are willing to settle multimillion dollar lawsuits rather than undergo a costly and burdensome e-discovery process – despite being completely innocent of the claims against them – it is hard to deny that the situation has reached a critical mass.

Fortunately, I believe it is not too late turn this ship around. We can begin to defensibly delete significant portions of our information repositories. We can classify and manage our most critical business information in complete compliance with internal and external requirements. We can finally put an end to burdening our information workers with time consuming records management responsibilities. And we can do it all by fully automating all lifecycle management processes across all of our information throughout the organization.

_____________________________________________

I recently asked Martin Garland, President of International Operations and co-founder of Concept Searching, if an Automated Information Lifecycle Management model was a realistic approach given the technology required and the cultural changes the Records Management community would have to accept to make it possible.

“Yes, it is.  In fact, we’ve been preaching that for the last four or five years”, Martin told me. “And we’ve been doing what we’ve been preaching the last four years or so with real-world customers.”

“So an AILM model of enterprise records management is attainable?”

“Absolutely. The proof is in the pudding, Don. We are actually doing this type of work right now, whether it be records, whether it be sensitive information, whether it be any type of information [the customer] wants to manage, lock down, identify… In my briefings, I always talk about the human factor and how the problem of information and content management is the human factor. You need to take the human factor out of the equation. But not at the expense of automation without human knowledge. We’ve taught that for the last five or six years in all of our briefings.”

Geoff Bourgeois, Chief Technology Officer at Acaveo, agrees. “I think the number of organizations out there right now [that are implementing an AILM model across the enterprise] is pretty low. Probably something like less than 1% – and you could probably still shoot some holes in their claims, as well”, Geoff said when we spoke recently.

“Oh, of course, Geoff”, I told him.  “I’m not suggesting any solution will ever be completely bullet proof. That’s just not realistic. But what I’m asking is can a reasonably comprehensive AILM model be realistically implemented in good faith across the enterprise?”

Geoff thinks it can. “A good message to get out there, Don, is this: ‘Folks, the toolsets, the underlying infrastructure, the technology is there to automate these processes. It’s just a matter of us understanding it, getting organized around it and using it.”

So if respected industry thought leaders like Martin and Geoff are right and an AILM model of records management is possible, what’s stopping us from doing it right now? In the next installment of this series, I’ll discuss some of the barriers to implementing a solution based on an AILM model and what needs to be done to work through them.

AILM ModelHere’s a brief conversation I recently had with a fellow records management professional:

Me: “I don’t implement manual records management processes anymore because of where I live.”

Other Guy: “Where do you live?”

Me: “The 21st Century.”

___________________

Let’s start the New Year out by being honest with ourselves. Things are a mess out there. Content growth is exploding at unprecedented, unsustainable rates. Storage devices are filling up faster than we can add them to our networks. E-discovery costs are forcing even innocent parties to settle lawsuits rather than engage in protracted and expensive discovery responses. And nobody – and let me be clear about this: nobody – is defensibly deleting information.

This is happening everywhere, across every industry and in every company regardless of the content management solution the organization is (or isn’t) using. Clearly those of us in the Records and Information Management profession have not been providing our customers with information management solutions that meet their needs. And I believe the only way we can change that is to fundamentally change the methods we use to manage the information lifecycle to focus on a singular goal: the complete automation of the entire records management process across the organization.

I call this the Automated Information Lifecycle Management model (and I fully expect to be accepting my Noble Prize for its creation this time next year in Oslo, Norway – if you want to book your travel plans early.)

Given the technology at the time, the Automated Information Lifecycle Management (AILM) model would not have been possible only a few years ago, but I now believe it is not only a possibility, but an absolute imperative if we have any hope of reversing the last two decades of over-retention and over-preservation.

Over the next few months I will be posting a multi-part series of articles describing the AILM strategy and how it applies to various aspects of records management, including email, social networking, the cloud, e-discovery and more. This installment – Part 1 of the Series – will explain the fundamental implications of applying the AILM across the enterprise.

[Note: With a few exceptions, my professional practice over the last several years has focused almost exclusively on records management in a Microsoft ecosystem. And, of course, this blog has focused on records management in SharePoint (hence, the rather obvious name SharePointRecordsManagement.com). But it is important to note that I also have many years of experience with almost every other major ECM/RM product on the market and unless otherwise stated, the methods I describe in this series are not limited to a Microsoft environment, but can be applied anywhere.]

___________________

For many records management professionals, the thought of automating every information lifecycle management process across an organization is nothing short of heresy. This is a 20th Century, paper-based-world mind set and we are going to have to move beyond it. Right now, technology exists that allows you to automate every phase of the information lifecycle. And it can be done easily, cheaply and in complete compliance with every organization’s internal and external requirements.

There are, of course, a number of significant implications that arise from applying the AILM model, though I believe all of them will result in a net positive for the organization:

  • Easily the biggest effect will be felt by the organization’s end users. Information workers – who have long been expected to participate in the records management process by applying burdensome metadata, analyzing complex retention policies, classifying content against complicated file plans, and manually ‘declaring’ records – will suddenly be free of these onerous responsibilities and able to spend more time on tasks directly related to their lines of business.
  • The AILM model will have a profound effect on the organization’s records management practice, making it far easier to manage, less costly to maintain and, most importantly, a great deal more efficient – providing the level of defensible disposition necessary to finally turn the tide on the explosive growth of over-retained information.
  • IT departments will rejoice. In many organizations IT staff have been forced into making impossible decisions: keep adding and maintaining storage for more and more data at higher and higher cost or begin to destroy information irrespective of its proper retention and disposition requirements. (I personally know people who have chosen the latter and they lie awake at night fearing the inevitable lawsuit that requires them to produce information they know is no longer there.) With an AILM solution implemented, the responsibility of making these no-win decisions will finally be lifted from IT’s over-burdened shoulders.
  • Corporate Legal departments will be equally as delighted. AILM will significantly reduce the cost and burden of e-discovery, while simultaneously ensuring the completeness of discovery responses.
  • And lastly, companies will finally have the technology in place that will enable them to implement retention and disposition policies on all of their information across the entire organization, ultimately allowing them to put an end to the pointless and counter-productive practice of distinguishing ‘records’ from ‘non-records’.

___________________

The time for the Automated Information Lifecycle Management model is now. (OK, technically, it was yesterday, but there isn’t much we can do about that.) In the next installment in this series I will be talking to two industry visionaries, Martin Garland, CEO of Concept Searching and Geoff Bourgeois, CTO of Acaveo about how their products support the AILM model and how some of their customers are already implementing it.

Box RMDespite the way things might appear, I am not completely in love with SharePoint. It is far from perfect, particularly when it comes to records management. I do admit to liking it, though. Certainly more than the other ECRM solutions I’ve worked with. (And I’ve worked with – and sometimes, for – them all.)

But in my heart I’m a records manager. Sad though it may be to the uninitiated, I am passionate about efficiently managing information through its lifecycle. And I’m always looking for innovative, next-generation solutions to meet the seemingly endless parade of records management challenges brought on by the ever quickening march of information technology. Regardless of whether the solution involves SharePoint or not.

That’s why I was so intrigued when I heard that Box – a content management and file sharing service, created and maintained entirely in the cloud – announced the forthcoming release of their first set of records management features. And it is also why I jumped at the opportunity to speak recently with Annie Pearl, Senior Product Manager at Box, about what we can expect to see when these new features become available.

Here’s what Annie had to say:

SPRM: Thank you for talking with us today, Annie.

Annie Pearl: My pleasure, Don.

SPRM: Where are you calling in from, California?

Pearl: Yep, I’m here at Box headquarters in Los Altos.

SPRM: Lucky you. Tell us about your role at Box?

Pearl: I’m the Senior Product Manager on the Box Enterprise Team, so a lot of the features I work on are in the world of governance, security, workflow – many of tools we have in our Administrative Console. And so, one of those areas, especially around governance, is the notion of retention management, records management.

SPRM: Right, so is that the category you put these new features into, records management? Or do they fall into information governance? Or is it something else altogether?

Pearl: I would have to say information governance. Mainly because, as you know, records management means a lot of things that are probably a lot more complex than we are doing now. All we are really trying to do is meet the regulatory needs and mandates of our customers. I wouldn’t go as far as to call it records management. I would call the specific features retention management. But I think it all flows into a bucket around information governance, where we have a number of other features like Policy Manager, which gives you control over the types of content that can be uploaded. We have e-discovery features to allow you to support a defensible discovery solution, and so I put retention management into that same bucket, which, sort of rolls up into information governance.

SPRM: What will these new features include?

Pearl: It is essentially being able to apply retention and disposition policies to certain objects inside of Box.

SPRM: How will those policies be applied?

Pearl: It will be folder based, with the ability to set a blanket default retention policy across your entire enterprise for any folder. So you can say, for instance, ‘The default retention across all my content is three years, on the regular course of business content’.  And then have specific folders that we will want to retain longer.

SPRM: When the content has met its retention period and is destroyed pursuant to the applied policy, would that be forensic-level destruction where the content is no longer recoverable? Or is it closer to the notion of simply hitting the delete key and releasing the space on the drive where the content resides?

Pearl: No, it is truly forensic deletion. It’s gone from the system completely.

SPRM: Terrific. That level of destruction is critical.

So it sounds like you’ve created a location-based model for records management, which I’m very happy to hear. It’s always been our argument that that’s the only realistic way to manage the lifecycle of all your content across the entire organization.

Pearl: Yes, exactly.

SPRM: Great. So what date do the retention policies use to trigger the retention period?

Pearl: Retention will be are triggered off of upload date.

SPRM: Got it. Are event based retention triggers something you are considering?

Pearl: Yes. One way to do event-based retention with our current product is use the ability to extend retention, so if the retention period is complete and the record is up for disposition, and the end user or the administrator wants to, they can extend the retention period on an object. So one way you could do event-based retention would be to basically retain until the end of the retention period and if the event still hasn’t taken place – let’s say it’s account documentation for a financial services customer and they’re still a customer – so we have to retain it for another period of time. And once they became a customer even longer, you would be able to extend the retention policy based on however much time you would need after the event has taken place.

So it’s kind of a manual event-based retention trigger today, but we are looking into the future at the potential of having it automated.

SPRM: Are you doing this from a centralized location? Are you managing these retention and disposition policies in a centralized way or will the end user be required to navigate out to the individual folders and set the policy there?

Pearl: No, that’s not necessary. In the Box Administrative Console we have an area where we can manage different policies around your content and one of those policies will be for setting the retention. So centralized in that location is where you essentially setup the policies. If it’s a default policy you can set it to apply to all folders today and going forward and any new content created will automatically inherit that policy.

Alternatively, if you wanted to specify specific folders you could still use the centralized Administrative Console to create another policy. There’s a folder picker that will allow you to find whatever folder or multiple folders you want to apply that policy to. And it’s all centrally managed in our Admin Console.

SPRM: So what are your plans for maintaining these records long-term? How would you expect to maintain a record for 25 years, for instance?

Pearl: There are a couple of different answers to that. We have unlimited storage, so the notion of needing to manage long term storage from a cost perspective is not an issue for our customers. So if you wanted to assign a 25 year retention to a piece of content, you could.

It’s more of an issue if the end user didn’t want to interact with that content anymore they could move it to trash, which is essentially an area that is not in the ‘all-files, all-folders’ view, but it would still be retained by the system for however long the retention policy is.

In addition to that, in this same area I’ve been talking about – the centralized Administrative Console – we have what we call the Content Manager, which essentially allows you to have a view into all the content in your enterprise and you can do different filtering on that view to find retained content or content that’s subject to a legal hold or really whatever you want.

You can use this view into all content inside your enterprise to manage records as in ‘show me all records that are currently under retention’ or ‘show me all records that are coming up for disposition within a certain time period’. You can already manage all that in our Administrative Console.

SPRM: Can you also apply a legal hold from there?

Pearl: Today, we have the ability to apply a hold at the enterprise level and the user level and we have tools today to be able to manage legal holds. In which case, of course, a legal hold will always trump a disposition action that was the result of a retention policy expiring.

SPRM: Would you be able to manage e-discovery from the Administrative Console or is that a different process?

Pearl: Yes, absolutely. We have all the tools for our e-discovery support in the Administrative Console, so that’s where you can audit all the events that take place on any piece of content or any user or any folder…you can do queries to find relevant content to a lawsuit. You can then preserve that content inside of Box. And we have tools to essentially collect it and pull it out of Box and put it into whatever e-discovery tool you might be using, along with all the other content sources in your enterprise.

We really approach e-discovery with the thought that we need to provide the tools that support a defensible discovery process, but we are not an e-discovery vendor. And so we partner with all the leading e-discovery vendors and we feel that our responsibility is to provide the ability to identify content, to preserve it and collect it out of Box.

So there are really the three main areas where you get the ability to do advanced queries on the content, you can then hold that content and preserve it and you can collect it out of our APIs or export it out of our Administrative Console, as well.

SPRM: This has been really enlightening, Annie. It’s exciting to hear about these new features Box is implementing. We really believe there are a lot of records managers out there who are ready to embrace the cloud and all its potential. And they’re looking for simple and efficient solutions that allow them to effectively manage the information lifecycle without all the bloated functionality that weighs down so many on-premises legacy solutions.

Pearl: Yeah, and I think that’s the goal. Every time you start to include these types of features there’s a whole universe of functionality you could go after, but the challenge is to distill that down to the most common denominator that still provides enough value for it to be a viable option for whatever you are trying to get done.

SPRM: Well, you certainly seem to be going at it the right way. Please come back and let us know how the initial roll out goes.

Pearl: I’ll be sure to do that, Don. Thanks.

 

Follow

Get every new post delivered to your Inbox.

Join 426 other followers