AILM ModelHere’s a brief conversation I recently had with a fellow records management professional:

Me: “I don’t implement manual records management processes anymore because of where I live.”

Other Guy: “Where do you live?”

Me: “The 21st Century.”


Let’s start the New Year out by being honest with ourselves. Things are a mess out there. Content growth is exploding at unprecedented, unsustainable rates. Storage devices are filling up faster than we can add them to our networks. E-discovery costs are forcing even innocent parties to settle lawsuits rather than engage in protracted and expensive discovery responses. And nobody – and let me be clear about this: nobody – is defensibly deleting information.

This is happening everywhere, across every industry and in every company regardless of the content management solution the organization is (or isn’t) using. Clearly those of us in the Records and Information Management profession have not been providing our customers with information management solutions that meet their needs. And I believe the only way we can change that is to fundamentally change the methods we use to manage the information lifecycle to focus on a singular goal: the complete automation of the entire records management process across the organization.

I call this the Automated Information Lifecycle Management model (and I fully expect to be accepting my Noble Prize for its creation this time next year in Oslo, Norway – if you want to book your travel plans early.)

Given the technology at the time, the Automated Information Lifecycle Management (AILM) model would not have been possible only a few years ago, but I now believe it is not only a possibility, but an absolute imperative if we have any hope of reversing the last two decades of over-retention and over-preservation.

Over the next few months I will be posting a multi-part series of articles describing the AILM strategy and how it applies to various aspects of records management, including email, social networking, the cloud, e-discovery and more. This installment – Part 1 of the Series – will explain the fundamental implications of applying the AILM across the enterprise.

[Note: With a few exceptions, my professional practice over the last several years has focused almost exclusively on records management in a Microsoft ecosystem. And, of course, this blog has focused on records management in SharePoint (hence, the rather obvious name But it is important to note that I also have many years of experience with almost every other major ECM/RM product on the market and unless otherwise stated, the methods I describe in this series are not limited to a Microsoft environment, but can be applied anywhere.]


For many records management professionals, the thought of automating every information lifecycle management process across an organization is nothing short of heresy. This is a 20th Century, paper-based-world mind set and we are going to have to move beyond it. Right now, technology exists that allows you to automate every phase of the information lifecycle. And it can be done easily, cheaply and in complete compliance with every organization’s internal and external requirements.

There are, of course, a number of significant implications that arise from applying the AILM model, though I believe all of them will result in a net positive for the organization:

  • Easily the biggest effect will be felt by the organization’s end users. Information workers – who have long been expected to participate in the records management process by applying burdensome metadata, analyzing complex retention policies, classifying content against complicated file plans, and manually ‘declaring’ records – will suddenly be free of these onerous responsibilities and able to spend more time on tasks directly related to their lines of business.
  • The AILM model will have a profound effect on the organization’s records management practice, making it far easier to manage, less costly to maintain and, most importantly, a great deal more efficient – providing the level of defensible disposition necessary to finally turn the tide on the explosive growth of over-retained information.
  • IT departments will rejoice. In many organizations IT staff have been forced into making impossible decisions: keep adding and maintaining storage for more and more data at higher and higher cost or begin to destroy information irrespective of its proper retention and disposition requirements. (I personally know people who have chosen the latter and they lie awake at night fearing the inevitable lawsuit that requires them to produce information they know is no longer there.) With an AILM solution implemented, the responsibility of making these no-win decisions will finally be lifted from IT’s over-burdened shoulders.
  • Corporate Legal departments will be equally as delighted. AILM will significantly reduce the cost and burden of e-discovery, while simultaneously ensuring the completeness of discovery responses.
  • And lastly, companies will finally have the technology in place that will enable them to implement retention and disposition policies on all of their information across the entire organization, ultimately allowing them to put an end to the pointless and counter-productive practice of distinguishing ‘records’ from ‘non-records’.


The time for the Automated Information Lifecycle Management model is now. (OK, technically, it was yesterday, but there isn’t much we can do about that.) In the next installment in this series I will be talking to two industry visionaries, Martin Garland, CEO of Concept Searching and Geoff Bourgeois, CTO of Acaveo about how their products support the AILM model and how some of their customers are already implementing it.

Box RMDespite the way things might appear, I am not completely in love with SharePoint. It is far from perfect, particularly when it comes to records management. I do admit to liking it, though. Certainly more than the other ECRM solutions I’ve worked with. (And I’ve worked with – and sometimes, for – them all.)

But in my heart I’m a records manager. Sad though it may be to the uninitiated, I am passionate about efficiently managing information through its lifecycle. And I’m always looking for innovative, next-generation solutions to meet the seemingly endless parade of records management challenges brought on by the ever quickening march of information technology. Regardless of whether the solution involves SharePoint or not.

That’s why I was so intrigued when I heard that Box – a content management and file sharing service, created and maintained entirely in the cloud – announced the forthcoming release of their first set of records management features. And it is also why I jumped at the opportunity to speak recently with Annie Pearl, Senior Product Manager at Box, about what we can expect to see when these new features become available.

Here’s what Annie had to say:

SPRM: Thank you for talking with us today, Annie.

Annie Pearl: My pleasure, Don.

SPRM: Where are you calling in from, California?

Pearl: Yep, I’m here at Box headquarters in Los Altos.

SPRM: Lucky you. Tell us about your role at Box?

Pearl: I’m the Senior Product Manager on the Box Enterprise Team, so a lot of the features I work on are in the world of governance, security, workflow – many of tools we have in our Administrative Console. And so, one of those areas, especially around governance, is the notion of retention management, records management.

SPRM: Right, so is that the category you put these new features into, records management? Or do they fall into information governance? Or is it something else altogether?

Pearl: I would have to say information governance. Mainly because, as you know, records management means a lot of things that are probably a lot more complex than we are doing now. All we are really trying to do is meet the regulatory needs and mandates of our customers. I wouldn’t go as far as to call it records management. I would call the specific features retention management. But I think it all flows into a bucket around information governance, where we have a number of other features like Policy Manager, which gives you control over the types of content that can be uploaded. We have e-discovery features to allow you to support a defensible discovery solution, and so I put retention management into that same bucket, which, sort of rolls up into information governance.

SPRM: What will these new features include?

Pearl: It is essentially being able to apply retention and disposition policies to certain objects inside of Box.

SPRM: How will those policies be applied?

Pearl: It will be folder based, with the ability to set a blanket default retention policy across your entire enterprise for any folder. So you can say, for instance, ‘The default retention across all my content is three years, on the regular course of business content’.  And then have specific folders that we will want to retain longer.

SPRM: When the content has met its retention period and is destroyed pursuant to the applied policy, would that be forensic-level destruction where the content is no longer recoverable? Or is it closer to the notion of simply hitting the delete key and releasing the space on the drive where the content resides?

Pearl: No, it is truly forensic deletion. It’s gone from the system completely.

SPRM: Terrific. That level of destruction is critical.

So it sounds like you’ve created a location-based model for records management, which I’m very happy to hear. It’s always been our argument that that’s the only realistic way to manage the lifecycle of all your content across the entire organization.

Pearl: Yes, exactly.

SPRM: Great. So what date do the retention policies use to trigger the retention period?

Pearl: Retention will be are triggered off of upload date.

SPRM: Got it. Are event based retention triggers something you are considering?

Pearl: Yes. One way to do event-based retention with our current product is use the ability to extend retention, so if the retention period is complete and the record is up for disposition, and the end user or the administrator wants to, they can extend the retention period on an object. So one way you could do event-based retention would be to basically retain until the end of the retention period and if the event still hasn’t taken place – let’s say it’s account documentation for a financial services customer and they’re still a customer – so we have to retain it for another period of time. And once they became a customer even longer, you would be able to extend the retention policy based on however much time you would need after the event has taken place.

So it’s kind of a manual event-based retention trigger today, but we are looking into the future at the potential of having it automated.

SPRM: Are you doing this from a centralized location? Are you managing these retention and disposition policies in a centralized way or will the end user be required to navigate out to the individual folders and set the policy there?

Pearl: No, that’s not necessary. In the Box Administrative Console we have an area where we can manage different policies around your content and one of those policies will be for setting the retention. So centralized in that location is where you essentially setup the policies. If it’s a default policy you can set it to apply to all folders today and going forward and any new content created will automatically inherit that policy.

Alternatively, if you wanted to specify specific folders you could still use the centralized Administrative Console to create another policy. There’s a folder picker that will allow you to find whatever folder or multiple folders you want to apply that policy to. And it’s all centrally managed in our Admin Console.

SPRM: So what are your plans for maintaining these records long-term? How would you expect to maintain a record for 25 years, for instance?

Pearl: There are a couple of different answers to that. We have unlimited storage, so the notion of needing to manage long term storage from a cost perspective is not an issue for our customers. So if you wanted to assign a 25 year retention to a piece of content, you could.

It’s more of an issue if the end user didn’t want to interact with that content anymore they could move it to trash, which is essentially an area that is not in the ‘all-files, all-folders’ view, but it would still be retained by the system for however long the retention policy is.

In addition to that, in this same area I’ve been talking about – the centralized Administrative Console – we have what we call the Content Manager, which essentially allows you to have a view into all the content in your enterprise and you can do different filtering on that view to find retained content or content that’s subject to a legal hold or really whatever you want.

You can use this view into all content inside your enterprise to manage records as in ‘show me all records that are currently under retention’ or ‘show me all records that are coming up for disposition within a certain time period’. You can already manage all that in our Administrative Console.

SPRM: Can you also apply a legal hold from there?

Pearl: Today, we have the ability to apply a hold at the enterprise level and the user level and we have tools today to be able to manage legal holds. In which case, of course, a legal hold will always trump a disposition action that was the result of a retention policy expiring.

SPRM: Would you be able to manage e-discovery from the Administrative Console or is that a different process?

Pearl: Yes, absolutely. We have all the tools for our e-discovery support in the Administrative Console, so that’s where you can audit all the events that take place on any piece of content or any user or any folder…you can do queries to find relevant content to a lawsuit. You can then preserve that content inside of Box. And we have tools to essentially collect it and pull it out of Box and put it into whatever e-discovery tool you might be using, along with all the other content sources in your enterprise.

We really approach e-discovery with the thought that we need to provide the tools that support a defensible discovery process, but we are not an e-discovery vendor. And so we partner with all the leading e-discovery vendors and we feel that our responsibility is to provide the ability to identify content, to preserve it and collect it out of Box.

So there are really the three main areas where you get the ability to do advanced queries on the content, you can then hold that content and preserve it and you can collect it out of our APIs or export it out of our Administrative Console, as well.

SPRM: This has been really enlightening, Annie. It’s exciting to hear about these new features Box is implementing. We really believe there are a lot of records managers out there who are ready to embrace the cloud and all its potential. And they’re looking for simple and efficient solutions that allow them to effectively manage the information lifecycle without all the bloated functionality that weighs down so many on-premises legacy solutions.

Pearl: Yeah, and I think that’s the goal. Every time you start to include these types of features there’s a whole universe of functionality you could go after, but the challenge is to distill that down to the most common denominator that still provides enough value for it to be a viable option for whatever you are trying to get done.

SPRM: Well, you certainly seem to be going at it the right way. Please come back and let us know how the initial roll out goes.

Pearl: I’ll be sure to do that, Don. Thanks.


Penguins as In-place RecordsOver the last few weeks I’ve had a number of interesting offline conversations with other records management professionals about some comments I made on ‘in-place records management’ in my previous post.  So I thought it would make sense to publically expand on those comments in a new article and hopefully provide some clarity to some of the statements I made.

In my post on upcoming new records management features in Box, I stated that in-place records management ‘simply doesn’t work’.  Let me try to clarify that statement.  What I meant by that is ‘in-place records management simply doesn’t work’.  It never has and it never will.  It may sound good on paper, especially to you academic types and records management theoreticians, but in reality the concept is fundamentally flawed. And here’s why.

If you’ve read this blog for any length of time, you’d know I am extremely hesitant to compare the electronic records management methodologies of today to paper-based records management practices of the past.  In fact, I have been very vocal in my belief that our attempts at managing electronic content the same way we managed paper records is the primary cause of the staggering records and information management chaos so many organizations face today. So naturally, I’m about to compare the electronic records management methodologies of today to paper-based records management practices of the past…but I do so with great reluctance and only because in this one particular instance, the comparison is quite valid.

Think back to about 20 or 25 years ago, when life was a lot simpler and we managed virtually all of our content on paper. We would bang out stacks and stacks of paper records, which we would store in desk drawers or file cabinets or in boxes on shelves somewhere in our office. We termed this the ‘Active’ records retention stage because we knew there was a reasonable chance that we would reference this material in the near future and it made sense to maintain it someplace close by.

Then, periodically, we would purge our active paper records by transferring the records with long-term storage requirements to onsite or offsite storage facilities and destroying the remaining records that had met their retention requirements while in the active retention stage. The records transferred to the storage facility would remain there in the ‘Dormant’ retention stage for as long as was required.

In-place records management is the electronic equivalent of eliminating a dormant stage in the paper records management lifecycle. In paper records management terms, it would be the same as leaving your records, not just in cabinets and desk drawers, but scattered haphazardly in every nook and cranny across your organization. And here’s the kicker: in order to meet legitimate records management requirements, some of those records would need to remain there for decades. Or worse, in some cases, permanently.

Ask yourselves this: ‘Is my IT department prepared to maintain and support records stored in-place in project sites, team sites, wikis, blogs, etc., etc., distributed with virtually no control across the entirety of the enterprise for the next 10 years?  How about 20 years?  How about 100?’

The answer is, of course, no.  But this is the expectation – actually, the reality – of in-place records management.

The truth is, in-place records management is not really records management, at all.  It is simply changing a few properties on a few documents and declaring them ‘records’.

But look, if you don’t agree with me, I understand.  A lot of vendors (including Microsoft), consultants and marketers have invested a great deal of time and money in creating, distributing and promoting in-place records management features. And I’m only one guy, jumping up and down and shouting to anyone who will listen to me like an escaped mental patient.  But do yourself a favor and listen to our old friend, John Holliday – someone who quite literally wrote the book on SharePoint records management – as he gives his opinion on in-place records management in this short YouTube video:


Box Storm CloudsAt the very end of the original Terminator movie, Sarah Conner is parked at a gas station looking out toward the horizon where she sees dark clouds forming. A young boy sitting on a bike nearby notices her stare, looks at her and says, ‘A storm’s coming.’ Sarah replies, ‘You have no idea…’

It’s one of the all-time great closing movie scenes and you don’t have to be a UCLA film major to understand writer/director James Cameron’s characteristically ham-fisted subtext. The boy looked at the clouds and saw an actual storm approaching. Sarah looked toward the horizon and saw the coming Robot Apocalypse.


Clearly I have a vested interest in SharePoint. I think it’s a terrific platform that offers almost limitless possibilities for solution development. And I enjoy working with it and watching it evolve with each new release. But over the last couple of years I’ve been watching with increasing interest what they are doing at Box.

If you’ve never used Box, it is a free (for personal use), entirely cloud based content management and file sharing service. Box is extremely user friendly with a wonderful, highly intuitive interface that makes collaboration simple and almost fun to use. And it is gaining new users at an impressive rate.

To be fair, part of the appeal of Box may be due to SharePoint fatigue. After years of working within a virtually unchanged SharePoint user experience, Box can start to look like the shiny new toy that made your old toys look boring. Essentially, Box is Buzz Lightyear and to SharePoint’s Woody.

But I honestly believe it goes deeper than that. SharePoint has not been around as long as most of the large enterprise content and records management products that it has traditionally competed with. Yet in that relatively short period of time, it has managed to capture the majority of the ECRM market. It did this by offering basically the same features as the other products at a lower total cost of ownership.

But what happens when the market realizes that many of those features become obsolete as technology advances? Or worse, the features were never necessary to begin with? Take ‘in-place’ records management, for example. Ten years ago, the legacy ECRM products all included some form of in-place records management. Not wanting to be left out, Microsoft included in-place records management functionality in its release of SharePoint 2010.

Now I have argued for many years that in-place records management fails as an enterprise records management solution model. It may seem like a good idea, but it simply doesn’t work. In my 20-plus year career in the industry, I have never seen an in-place records management solution deployment succeed. And it will fail in SharePoint, just as it has failed when implemented using one of SharePoint’s traditional competitors.

But SharePoint’s competitors had in-place records management, so Microsoft added it to SharePoint. And over the years, this process was repeated over and over again. In many cases, it seemed as if Microsoft would look at capabilities in other ECRM products and simply duplicate them in SharePoint without considering whether or not the capabilities added real value to its users. Add the third-party SharePoint add-on products that so many vendors insist you absolutely, positively can’t do without, and SharePoint can start to look more like a system for managing the Hadron Particle Accelerator than enterprise content and records. (And it can cost about as much, too.)

Given that we are already awash in information and its growth will continue to explode for the foreseeable future, complicated, burdensome ECRM solutions of any kind will begin to collapse at an accelerated rate…and that is what I find so compelling about Box.

Unlike its competitors, Box isn’t burdened with decades of legacy features – most of which were developed for on-premises solutions and many of which no longer provide much value – which it must continue to support in the cloud. Instead, Box is free to create capabilities using state-of-the-art technologies that are simple, intuitive and enjoyable to use.

And that’s just what they’ve done. Several years ago Box predicted mobile devices would become a critical component of every content management solution. So the company made a strategic decision to invest in developing innovative methods for connecting to Box content using any device anywhere in the world. Today, no other ECRM product can match the mobility capabilities that Box provides.


Despite what you hear from its competitors, you can manage records quite effectively in SharePoint – on premises or in the cloud. And you can do it using next generation records management methodologies that leverage the most advanced technologies available today. But if organizations are ever going to successfully implement SharePoint records management in a cloud environment, the User Community, the Microsoft partner ecosystem, and, in particular, Microsoft itself, must come to grips with the fact that many of the records management features it struggled to implement on-premises will fail miserably in the cloud.

Here’s why this is important. I’ve admired what the folks at Box have done, but I am a Records Management professional. My practice is focused on managing information through its entire lifecycle. And Box hasn’t included any features that provide even the most fundamental of records management capabilities. So, while I may like the product, it hasn’t been one I can recommend to my customers.

But that may be about to change. Earlier this month at its annual conference in San Francisco, Box announced that it was releasing new retention management and auto-classification features. These features aren’t due to be released for a few months, so we will have to wait a little bit to fully understand what they are capable of doing. But if they function as the company describes, they would provide Box with true next generation records management functionality that would put its product next to – or in front of – every other cloud solution on the market.


These are exciting times for Box and potentially challenging times for Microsoft. They have never ignored Box. They recognize the product could be a threat to SharePoint’s market dominance. But that small storm they once saw coming over the horizon might have just become a Robot Apocalypse.

We will know soon enough…

Christensen InterviewQuentin Christensen is a Microsoft Program Manager for Archiving, eDiscovery, and Devices. In SharePoint 2010 and 2013 he worked on key records management features such as In-Place Records, retention policies, site policies, content organizer, CMIS, and the Records Center.

Included among Quentin’s very long list of responsibilities is overseeing eDiscovery and compliance in Office 365 and on premises Office server products including Exchange, Lync, and SharePoint.

Quentin was kind enough to chat with us recently and provide us with some of his thoughts and observations on information lifecycle management in Office 365 and SharePoint 2010 and 2013.

SPRM: Thanks for agreeing to talk with us today, Quentin.

Quentin: It’s my pleasure Don. What could be more fun than travelling to the Bahamas to answer some questions about records management? Although I was expecting the trees, sand, and water to be a little more… you know real.

SPRM: [Laughs] Sorry to disappoint, but sadly, we only work on a virtual beach here at SPRM.

So tell us a little about your role at Microsoft.

Quentin: I like to call myself the Emperor of eDiscovery and Ruler of Records Management. But the truth is even more grandiose than that. I work with a team of program managers and developers to build eDiscovery and compliance capabilities in Office 365, Exchange, SharePoint, Lync, and Yammer. I am still fighting with HR to change my job title to Emperor, but until then I am a Program Manager Lead.

I love talking to people about the challenges they face, imagining solutions that will solve their problems, and then working with a team to build it. I spend my time talking to customers, running research studies, designing user experiences and architecture, working with developers, attending lots of meetings, and sending lots of email. How much better does it get than that?

SPRM: That must be pretty satisfying work, Quentin.

We believe this is a critical time for Records and Information Management. Constant change is occurring across the industry. And given the widespread growth in cloud environments, this will likely continue for some time. How do you suggest records managers leverage SharePoint 2010 and/or 2013 to best navigate their organizations through all this change?

Quentin: Imagine how records management can be rather than how it used to be or how physical records are managed. If you are changing your records management solution, now is the time to revolutionize the way you do things. I hear about three common problems from people. 1. What about all the content users are creating in OneDrive and team sites? 2. Can I manage electronic records in SharePoint and if so how? 3. How do I bring my electronic and physical records together?

I suggest focusing on solving one problem at a time and I think #2 is the easiest. You can manage records in SharePoint. Microsoft has terabytes of electronic records in SharePoint and this is out of box. Forget about trying to train every user in your organization to manage records and retention schedules. You just can’t scale that way. Focus on structured content that a set of well-known people work with such as employment records and contracts. Provide a solution where the data is created in the Records Center or a way for it to go there for long term storage and disposition. Once you have that figured out, move on to #1 or #3.

At SharePoint Conference in March we announced 1TB site collections and unlimited size tenants in Office 365. Now you can use the full set of records management capabilities in Office 365 including search, content organizer, send to, and Records Centers. This enables you to create a records management solution that you can send content to, automate where it gets stored, apply a retention policy, and use search so people can quickly find records in the system.

With Office 365 we manage the servers and scale so you can add content on your own terms. I have seen some painfully slow and cumbersome records management systems. With Office 365 and using search to access records, you can find content in less than a second, and Microsoft has had 99.98+ uptime for the past few quarters so your records are always available.

The most successful records management solutions I have seen all focus on simplification and automation. KISS really applies here: Keep It Simple, Stupid. This will make it faster to get up and running so you are providing value, plus it will be easier to manage over the long term. Use simplified retention schedules and a small amount of content types with as little required metadata as possible. Reduce friction getting content into the system and rely on search to find it again. Make it as easy as possible for people to get content in and you will have a far more successful solution.

SPRM: Can you give us an example?

Quentin: I think Microsoft is a great example, back in 2009 our Records Management team looked at using various systems for their next gen Records Management system and they chose SharePoint. We had 3,000 plus retention categories, which would have meant 3,000 content types. What a pain to manage. We simplified and arrived at 65 content types. This drastically reduced management overhead and the decisions people had to make when adding content. We entirely use SharePoint out of box and rather than relying on complex disposition approval for hundreds of millions of items (can you imagine our team of 3 records managers doing that?) we just use automatic retention. On a quarterly basis we do a review and ensure the right things are happening with the broad policies. Search ties it all together so anyone can search for and find important records when they need them.

SPRM: So does this solution provide the accountability your Records Managers are looking for?

Quentin: Am I sipping a pina colada and taking a long walk on the beach right now? Of course it does. It’s their job is to ensure content is available when needed to meet compliance requirements and company needs. It’s their butt on the line! That is why my team and I take our jobs so seriously. I love it when people thank me and say I have helped them sleep easier at night. By simplifying to a small number of content types and policies, it will be easier to manage and tell people what the policy is. Then you can ensure items have the right policy and are kept and destroyed on schedule. With the built in SharePoint end user and eDiscovery search capabilities it is easy to find what you are looking for and the legal team can get data out that they need for investigations and litigation.

SPRM: Makes sense. So what records management strategies seem to be the most successful and how are they typically implemented?

Quentin: The most common strategy is multiple Records Centers where people send content there. SharePoint splits content across site collections and you want to keep your site collections between 100GB-1TB to have high availability. Surpassing 100GB is quite easy for a records management solution, so everyone I talk to is managing content with multiple Records Centers and using search to easily find what they are looking for. Fortunately, SharePoint has out of box functionality including search, taxonomy, content organizer, and content type syndication that makes it easy to distribute and manage this data across multiple site collections.

SPRM: How are they applying Information Management Policies?

Quentin: Most of the customers I talk to are the biggest and most complex, we are talking about hundreds of terabytes or even more than a petabyte of records so this may not be representative of what everyone is doing, but I have seen three common methods. The easiest is to use content type based policies. You define your content types separately, and the content type drives auditing and when the item should be deleted. The second method is folders, this makes more sense when you have complex retention requirements. When items are submitted or uploaded, metadata determines what folder the item should go into. Then folders will determine the policy and when the item should be deleted. This is used when you have a large number of retention policies and want to manage content with structured folders. The third and most complex is to use custom metadata on every single item along with retention formulas or custom code. For example all items have a required managed metadata field. The custom code checks what that metadata is, calculates the retention schedule, and then sets a date for when the item should be expired. I have only seen two customers follow this path and I would recommend the first two options.

SPRM: What are the most effective declaration strategies you are seeing?

Quentin: The Declaration of Independence was pretty effective in its day. You can use the declaration of independence strategy with your own records. Let people do what they want, yeah it sounds crazy but you just can’t make everyone do it the records managers way. Maybe I am going insane from spending too much time as a UX designer. I believe making things super simple, fast, and easy for the average guy is paramount to user adoption. To make the declaration of independence succeed, you need two options: automatic and manual declaration. Then people can be independent. The people who know what they are doing can choose to use the structured process of declaring a record or setting metadata that sends an item that is ready to be a record off to the records center. For all that other content created by people who don’t know or need to know about records, make it automatic with broad deletion policies.

SPRM: ‘The Records Declaration of Independence’, I like that… Does it require custom code?

Quentin: I have seen several records management deployments including Microsoft that are all out of box and doing this, so no it does not require custom code. If you absolutely need complex lifecycles such as events that occur at some period of time later or you are integrating physical records or scanning paper content into SharePoint you will need some custom code. There are also some great partners out there that you should first take a look at before building the missing piece of the puzzle yourself.

A great reason to avoid custom code is that you want to keep it simple for your users, don’t place the burden on them with 30 metadata fields and a complex lifecycle management process that bugs them with workflows to review everything stuff. Look for opportunities where you can make everything automatic or remove steps.

SPRM: As everyone knows, eDiscovery costs are continuing to skyrocket, so having an efficient eDiscovery process is more important than ever. Tell us a little about how Microsoft is using the newest eDiscovery features in SharePoint 2013 and Office 365 to manage discovery orders.

Quentin: If I can do a shameless plug here, I have posted some articles that give an overview of eDiscovery that you should check out. Microsoft is using the new 2013 eDiscovery features in SharePoint, Exchange, and Lync for all cases and internal investigations. There are two common situations we have. First is the civil litigation case. We use the SharePoint eDiscovery Center to preserve content with In-Place Hold for between 10-40 people, this makes the data immutable but allows users to edit and delete their content without ever knowing it is on hold. Then we use search to reduce the data set by 80% or even up to 95%. Once that is complete we export the data. For many cases we use an additional 3rd party search tool to do additional search and analysis, then we hand it off to our litigation support vendor that hosts our review tool and manages the process of legal review. Before we started using the eDiscovery Center in 2013, we had already done a great job streamlining our eDiscovery process. Now that we use the eDiscovery Center, it saves our eDiscovery team a lot of time – it went from 4 weeks to 2 weeks to do initial preservation, so we are better protected and saving a lot of time for the people that manage the eDiscovery process. The second situation is internal investigations. Once again we use the eDiscovery Center to do search and preview to hone the keywords and then use export to get the data out of the system and store it in PSTs and loose files so we can analyze it further and keep it as evidence.

SPRM: But eDiscovery costs are only part of the litigation problems organizations face. You can’t pick up a newspaper – assuming anyone actually reads newspapers anymore – without seeing charges of spoliation and the penalties and fines that go with it. How are your customers managing their information in ways that prevent claims of spoliation?

Quentin: Well I am no lawyer, I just play one on TV. But this is what really keeps the lawyers up at night. There are three things you want to keep in mind here. With SharePoint and Exchange 2013 you can do In-Place hold, with search queries you can render content immutable even if new content appears. If it matches the query, it will be preserved. This is all done behind the scenes, users can create, edit, and delete content and never even know it’s preserved. The second thing is keeping your house in order with great records management practices. If you have an organized and documented SharePoint deployment with structured records and metadata, it will be easier for the legal team to find what they are looking for when they need it and ensure it is preserved to prevent spoliation. The third is defensible disposition. If you follow a proper policy, then you can clean up that old data so you have less data around that can be discovered. Then you can add the hold features in SharePoint to stop the deletion process when a legal event occurs.

But let me warn you, if you tell your eDiscovery team about these new capabilities, they will keep bugging everyone until you get upgraded to SharePoint 2013 so they can use the new eDiscovery Center. I have seen it before.

SPRM: So are they doing this in SharePoint? And your customers are doing this now?

Quentin: Did you even need to ask? I talk to customers every single day that are using In-Place Hold as well as retention. We have thousands of customers in Office 365 initiating preservation and performing eDiscovery searches every month.

And we are making things even better. At SharePoint Conference we announced Document Deletion policies that let you apply broad document deletion policies to site collections and site templates. For example, you could set a policy to delete all documents 4 years after they are created in OneDrive for Business sites. This will be coming to Office 365 soon. The great thing is you can do both hold and deletion on SharePoint, Exchange, and Lync content on premises as well as in Office 365. Note that with Exchange 2013, you can archive Lync IM and a meeting data into the user’s Exchange mailbox where you can then perform eDiscovery.

SPRM: Wow, that’s a cool feature. Are these solutions the same in the cloud using Office 365 as they are in on premises using SharePoint 2010 or 2013?

Quentin: The eDiscovery improvement between 2010 and 2013 is a big leap forward, I actually did a comparison of Records Management features between versions that you may find interesting. With Office 365 we deliver new features to customers earlier so soon you will be able to perform eDiscovery search across all SharePoint sites and do broad document deletion policies. In general we will have consistency between on premises and Office 365 so the new Office 365 capabilities will ship in the next major on premises release in 2015.

SPRM: Terrific. Many organizations have laws and regulations that they must apply to the records they manage based on the country, state or region in which the record is stored. Given that it is often impossible to tell where your information is physically stored in a cloud-based solution, how are your customers resolving that problem?

Quentin: Office 365 stores your data within certain regions, for example if you are in Europe you will use our European datacenter. The trans-border flow of data outside of Europe is a big issue, so this actually meets the needs of many customers. To learn more see Privacy authorities across Europe approve Office 365 privacy commitments.

We are also seeing hybrid deployments where customers have records centers both on premises and in Office 365. You can use hybrid search to search both at the same time to have the best of both worlds.

SPRM: Well, we still have a lot we’d like to ask you, Quentin, but I think we’ve used up enough of your time. Will you come back sometime for a follow-up?

Quentin: Only if we can have the interview somewhere more tropical like Hawaii.

SPRM: Ha! I’ll see if that’s in the budget. In the meantime, how can our readers keep up with the latest developments in SharePoint and Office 365 records and information management?

Quentin: For new product announcements you can see the Office product blog. I have also been sharing my own thoughts on eDiscovery and Compliance on my own blog so you can check that out as well.

Also I am doing an AIIM Q&A webinar July 23rd called Take the FUD out of Implementing SharePoint – Just Ask the Folks at Microsoft. It isn’t every day you get to do a Q&A with an emperor so I suggest you attend.

SPRM: We’ll do that. Thanks, again, Quentin.

Quentin: You’re welcome.


QC Blog - SmallQuentin Christensen is Program Manager for Archiving, eDiscovery, and Devices at Microsoft. (For the record, his unofficial titles are The Emperor of eDiscovery, Czar of Compliance, and Duke of Document Management.) Quentin recently started a blog on his thoughts around compliance, eDiscovery and enterprise content management.

Among Quentin’s long list of responsibilities is records management in SharePoint and earlier today he released the first of a two-part series on the latest developments on the topic. You can find that post here.

Part Two of the series will provide a deeper dive into SharePoint records management with a focus on physical records, scale, DoD 5015.2, distribution, organization, and eDiscovery. It should be out in a day or two.

Incidentally, I will be interviewing Quentin soon for an article to be posted on this blog. Please let me know here or offline if you have specific questions you would like me to include in the interview.



[Editor’s Note: While I was able to attend SPC14 for the first two days and talk to a number of key SharePoint Records and Information Management thought leaders (both within and outside Microsoft), circumstances back home prevented me from staying long enough to attend any of the conference sessions. With that in mind, I’ve asked our good friend from the Netherlands, Eric Burger, if he would permit us to publish an English-language version of his SPC14 Records Management and Compliance summary that appears on his blog. Eric kindly agreed. Here’s what he had to report. – Don]

SPC 14 Having been treated to a one hour speech on the benefits of communication technology by former president Bill Clinton, SharePoint Conference 2014 kicked into high gear with a keynote by Jared Spataro, General Manager Product Marketing at Microsoft. He highlighted the remarkable growth of SharePoint, Office 365 and SharePoint Online sales.  However, subsequent speeches by amongst others, Julia White, whose live demos introduced innovations in Office 365 not previously shown, got the audience really loose.  A far-reaching integration of Office and social features called Office Graph, which breaches the traditional dividing lines between the domains documents, email, and social will be released in the coming months.  Elements of Yammer can be added so that an overall picture of interaction, trends, communication and importance of certain objects, people and communications is created on the fly.  The entirely new ‘Oslo’ concept, which allows the user to work from a visually very attractive, individually composed screen, has been developed to bring the working environment – documents, actions, people – intuitively to the user, to make him or her ‘work like a network’.

The last demonstrations in Monday’s keynotes were a pleasant surprise to all of us compliance-minded professionals. A brand new Compliance Center in Office 365 is the SharePoint- and Exchange dashboard for the compliance officer or record manager. Existing functions for security (encryption, data loss prevention, and information rights management), audit, e-Discovery and retention can now be activated, configured and controlled on tenant level. Previously, most of these functions were limited to configuration per site collection. Auditing in Office 365 has been extended with ‘viewing’, something that already existed for on prem. And audit reports – just a prototype in the demo – will be readable soon! Adding to the transparency trend, an ‘admin audit log’ will make Microsoft administrators’ steps traceable. These functions extend to Exchange, so compliance officers will have control of e-mail messages, can even put deleted messages on hold.

Also new are the ‘deletion policies’ in Exchange and SharePoint Online, by which specified categories of messages and documents can be disposed of after a certain time. This is not intended as a retention period applied to a process or content type (retention policies), but for example, allows e-mail notifications (meetings, tasks) to be removed automatically.  It struck me that the granularity of this functionality overlaps with new opportunities in e-Discovery (time-based hold) and retention policies, so that even the speakers occasionally came up with the same examples for the various functionalities.  We have to find the right scenario’s for each.
Once again, Microsoft added functionality to its legal hold features, extending hold with time-based hold. Holds can be applied from the Hold Center to any keyword, metadata element, webpage or item in SharePoint, Exchange, Lync and file-shares. Yammer is still excluded, though, and any relevant content from social should be exported to a file-share first to be added to a hold.

So this Compliance Center is a big step forward in centralized SharePoint information governance. On Tuesday, Astrid McClean, Senior Program Manager Information Protection, did an excellent job in explaining the Microsoft ‘in-place’ strategy and diverse functionalities brought together here. Check out her presentation (audio and slides).

Some of the 200+ sessions in this conference dug deeper into existing (mostly SP2010-) implementations of records management in SharePoint for large multinational companies.  Main lessons to take home there: yes, you can do records management in SharePoint, even for the biggest multinational corporations, including General Motors worldwide. And, yes, record management should be centrally managed with compelling governance at the highest level.

Daniel Harris in his General Motors presentation (‘Managing 130,000 Users’ Documents and Records – Making It Easy While Maintaining Control’) focused heavily on the issue of governance.  Not entirely out of the box SP2010, governance was maintained by a custom form for easy site provisioning. Security- and retention policies and metadata were all pre-arranged there. As only 25Gb site-collections are handed out, anything bigger asks for new registration.  Before provisioning is activated two dedicated owners, a half hour of training and confirmation of compliance rules is obligatory.  Automatic provisioning also creates record centers for each of these site-collections.  Also, a few end-users’ buttons were added to file documents as a record.

Examples in a presentation by Nishan DeSilva, Senior Director, Information Governance Microsoft, (‘SharePoint for Large Scale Records Management – Hundreds of Millions of Documents and Beyond ‘) confirmed:

– Large archives in SharePoint are not the problem, uncontrollable growth of site-collections is;
– Custom workflows might be added to out of the box rules, to meet organizations’ needs in site provisioning;
– Only by strict and (where possible) automated governance is a large records management implementation controllable.

Other best practices, on Metadata and Content Type Planning, came from Lori Gowin, Premier Field Engineer at Microsoft. After all, we know that content types are often at the core of records management processes in SharePoint. Some jeering erupted though when pen and paper or whiteboard were still to be regarded as best practice. And of course: MS Excel!  The audience expected a little more sophisticated tooling by now. Her overview highlighted a few key lessons:

– Do not change or remove any of the default content types in SharePoint – terrible things can happen – but make new types, derived from the standard;
– Plan in advance and make sure that the design has some room to grow;
– Keep the list of content types (not very clear to begin with in SharePoint) by formulating ‘categories’ as simple as possible;
– Use the content type hub to publish content types beyond the boundaries of site collections.

So where Office 365 was lacking robust records management functionality at the time of its 2011 introduction, it is now delivering the latest Microsoft technology, with SharePoint on premise falling behind. Code for the Compliance Center had just been tested shortly before SPC14 and will be shipping – added to the Office 365 environment automatically – in a couple of months.


Get every new post delivered to your Inbox.

Join 406 other followers