Findability as it Relates to Microsoft SharePoint and Enterprise Content Management

If you work in the area of technology, you might find the title of this article offensive.  I also started my career, many years ago, in the area of technology; and learning that technology alone can’t solve certain information management problems is difficult for many to swallow.

The information landscape has changed and the consumers of that information have usage expectations that grow every day.  What are the information consumer expectations I am referring to?  Let’s take one facet of Findability to explain what I mean; Search.  When you go to the Internet, you most likely use Microsoft Live, Google or Yahoo search facilities to find what you are looking for.  Your expectation is for that search engine to provide you relevant results (first or second page); and, you expect them quickly.

It is important to note that the scope of your search, and the expected results, are the Internet; meaning, return all information that can be found regardless of its source.  You didn’t have any preconceived notion the results would be of smaller scope; after all, you are searching the Internet.  Another important note to make is with regard to information accuracy.  I hope you didn’t put much faith into the accuracy of the information.  You may not have a clue who the author of that information is, how long it has been on the Internet or when it was last updated.  It is entirely up to you to determine information accuracy.

What did it cost you to produce those search results?  As an information consumer, it didn’t cost you a dime; so the first expectation is “search is free”!

Now move this same search expectation to Intranet content.  I’m sure you’ve heard it “I want Google quality search results on our Intranet”.  So you go to your Intranet, such as one built on Microsoft SharePoint technologies, execute a search and sure enough you see results that include every piece of content that matches your search term.

But wait!  That’s not what your users were expecting.  What they were really expecting was search results scoped to the context of their specific and immediate need.  Huh?  Hold on just a minute; they told us that Google search result quality was the objective.  Is it or is it not the real objective here?  We gave them the technology they begged us for and are still not happy.  Now what?

Let’s take a moment to dig a little deeper.  Not only were those users expecting relevant results within the scope of their specific contextual needs, now they expect them to be accurate.  They expect to be viewing the golden information; information the business uses to make formal decisions.

Okay, we start yelling “out of scope.”  You didn’t tell us about any of these expectations.  Microsoft Enterprise Search, Microsoft Live or even Google search can’t do this.  None of these technologies will magically understand the scope of every employees specific search need.  That’s an impossible task and we don’t have the technology to do it.  Oops… we just said it.  We don’t have the technology to solve this business problem.  Depression sets in…

Now we are back where I started; Tools and Technology Alone is not the Answer.  We have reached a point where the business user expectations have exceeded what the technology alone can accomplish.

I do want to note one last time that the art and science of Findability is multi-faceted and search covers only part of the big picture.  Agreed, it is a big part and at the forefront of information management needs, but only part.  I will be covering the other facets related to Findability at a later time.

Seems we have reached an insurmountable chasm; we have a business problem that technology alone can’t solve.  How large and deep is that chasm?  Well… that depends entirely on what your end users real expectations are.  You can also view this chasm as the gap between user expectations and a clear understanding of what it takes to get there.  Internet search engines such as Microsoft Live, Google and Yahoo have treated us very well.  However, they have lured us into a false sense of free; i.e. what it takes to produce results that are in alignment with user needs.   That false sense of security (free) comes from the belief that we can search billions of web sites and produce relevant results so it should be an extremely simple task to take a small fraction of that in our business environment and do even better!  Right?  Less information to search so the results will be better…  And best of all, it’s free!

So where is the gap in our thinking; where did this breakdown occur?  It all boils down to scope and context.  The user made the transition from Internet to Intranet without considering what happened during that transition.  They made that transition subconsciously imposing a significant amount of scope and context into their search expectations.  Believe it or not, it was this subconscious decision that causes the deepest of resentments, heated conversations and outcries of how bad our technology is with regards to search.

If you find that hard to believe, I invite you to survey business users in your organization.  Don’t set them up with any preconceived notions of outcome, simply ask two questions:

1.       After we roll-out SharePoint, is your expectation to easily search for any business information you need?

2.       Do you expect those search results to include only information that is up-to-date and accurate?

 I think you will find the majority answer “yes” to these two simple questions.  Those two questions alone, can explain the gap in what that user initially asked for and what they expected.

Let me fill you in on a tidbit of information:

1.       Search technology alone doesn’t know anything about scope or context.

2.       Search technology alone doesn’t know how to determine if information is up-to-date or accurate.

Sounds like I am talking in circles now; because we are back at step one.  Technology alone can’t do it.

Do I dare venture deeper into the real expectations our users aren’t telling us?  Before I do, I will say technology is not, in itself, the problem; this problem is a people problem.  It takes a well-balanced approach to this business problem that includes technology, process and people to solve.  And the injection of people into this approach causes all the disruption.  Why?  It’s back to the expectations and subconscious change in those expectations made during the transition between the Internet to Intranet.

We, somehow, first need to convince our business users there was a change and now they need to take time tell us all about it.  With all this talk about change and subconscious transitions, it sounds like we need to transition from IT to Psychiatry.   This, in itself, could be a monumental task.  How can we educate our users of this transition?  And, even if we are successful convincing them of this subconscious transition, how do we explain the amount of time it will take to understand what their real expectations are?

I am beginning a series of articles, and free webinars that will help you better understand how to solve these types of problems.  Enterprise Content Findability is a broad topic that spans many areas we have never had to deal with.  My intention is to ultimately help you understand how to close this chasm of untold expectations and produce Findability in your Enterprise Content Management solutions.  Findability is the ultimate goal of all Enterprise Content Management systems and it can be accomplished.  However, to produce a solution that promotes content discovery and relevant search results requires effort.  The amount of effort invested is proportional to the outcome; i.e. alignment of technology with business user expectations.

What else aren’t they telling us?

I hate to make this sound more complex but I have already opened this can of worms, so I may as well paint the big picture.

I hope you do take the time to understand what your business user expectations are.  It is only then you can put a plan together that will reduce the gap between what you have and what they want.

During your initial interview, you may also wish include in your questionnaire:

3.       Would you be surprised to learn the expectations you have came from searching the Internet and those expectations have little relevance with regards to searching our business information?

It has been my experience this question raises a defensive response.  The goal is to not upset your users but to begin helping them think about the subconscious change of expectations.  This is also a good time to help them understand what that change was.

Another important goal is to learn where each user is with regards to valuing information they work with each day.  This level of valuation doesn’t have to equate to some monetary value yet, just some level of importance.

4.       Let’s take a moment to value the information you need each day to be productive.  For the purposes of this valuation, we will use a scale of 1 to 5; where 1 means information is of no value to you at all and 5 means information is critical to your success each day.

The answer to this question may come as a no surprise.  Most will answer at the higher end of the scale.

The last is to gain some sense of the monetary value associated with their information.  If the individual you are interviewing has a budget you may wish to ask them:

5.       If I told you it could cost $100,000 to produce search results that meet your expectation, would you be willing to pay that out of your budget?

The answer to this question may also come as no surprise.  Most will say No!  Why?  Remember, they have the preconceived notion that search is free and can already produce these results.

Now that you are armed with information about perceptions and expectations, what should you do?  In a nutshell – you build a plan:

1.       Based on how your organization values information balanced with managed expectations, you educate them on what needs to be done to accomplish your goals.

2.       If you have succeeded at #1, you then need to educate them on the expertise required.

Balancing the Value of Information with Expectations

Information in your organization only provides real value once it is included in processes and delivered to people.  Let me provide you with an example of this.

Your sales department may be the group responsible for Customer Contracts.  Considering the evolution of a Customer Contract by only involving the sales department could lead you to a solution that is relatively simple to implement; i.e. Contract document creation, review, approval, signatures and archival.  All of the document management functions can easily be managed in SharePoint.  So where is the business value in managing this document?  There is some, such as version control, automation of review and approval with workflow and so on.

However, the real value is when you think outside the box and apply other processes and people.  Consider the following example.

Your marketing department would like to cross-sell a new product to existing customers who have purchased products since July 2007 with a value exceeding $100,000.  How would they produce a list of customers, based on past history (Customer Contracts) matching this criteria?

If your first thought is to search SharePoint, the search query could be quite complex.  It would need to include the following criteria:

·         Customer AND Contract AND 100000 and Modified Date > July 2007.

Could you create such a query?  Sure, not this same syntax, but it is possible.  The bad news is, the search will return all documents in your database that matches these criteria.  Huh?  But that is what I want, right?  Nope…  It will include all documents that meet these criteria.  Technology may have Statement of Work documents on project sites that contain the terms Customer, Contract and 100000.  You may also get IT Purchasing Vendor Contract documents that contain these terms.

Our user in marketing has added process to Customer Contracts and is expecting accurate results delivered within the scope of those documents only.  Technology does have the ability to produce accurate results.  However, to do so, the technology has to first understand scope.  In this example, the scope is Customer Contracts; we need to configure the technology so it understands which documents are specifically Customer Contracts.

Once we have configured the technology to understand the scope of Customer Contracts, we can limit the amount of information searched to include only that scope.  Fantastic!  That’s all there is to it?  No, it’s not…  What we have at this point is an understanding of which documents are of the type Customer Contract.  The next criteria the marketing department needs to search for is Customer Contracts with a value exceeding $100,000.  So let’s try our search again.  Now that we have limited our search to a scope that only includes Customer Contracts we can submit a search query that includes:

·         100000 and Modified Date > July 2007.

Okay, did we get the results we are looking for yet?  Most likely not.  This will now return all Customer Contracts that contain the number 100000 and a Modified Date > July 2007.  They’re very well could be Customer Contracts that were valued at $50,000 but contain other figures matching this criteria.  In addition, a customer could have purchased a product and paid for it 3 years ago but just updated the maintenance agreement 2 months ago; so the last modified date of the contract will reflect 2 months ago.  These documents would be considered false positives; false with regards to the business need but positive with regards to the specific search criteria.

To produce accurate results, we would also need a specific way of singling out Customer Contract documents with a specific sales value and date.  The most common approach to solving this is to use metadata.  Once we have configured the system to use this metadata and include it with each Customer Contract we can produce search results that add value to the marketing department.

It is my hope this simple, but common scenario, will help you understand the depth and breadth of knowledge needed to produce relevant search results.  The reality is, you most likely won’t know about the Marketing Department needs when you first work with and understand how Customer Contracts are used.  This is because you are assessing Customer Contract information by working with the Sales Department.  In doing so, the Sales Department will give you a myopic view of their specific process and contextual needs.  It is during this time you are at the greatest risk of building a solution that is not scalable to support the needs of other users in the future.

This is where an experienced Enterprise Information Architect (EIA) comes into the picture.  Such an individual on your team can help you take a step back, look at the bigger picture, and design an Information Map that lends itself to future needs.

An EIA may not be a competency available to your Intranet Team.  This is where the rubber hits the road with regards to how your organization values its information.  Without an experienced EIA, you can’t architect and design a solution that is in alignment with user expectations.  If your organization doesn’t see this value, you will have a difficult time obtaining the appropriate budget to complete the project.  It is at that point you need to find the balance between Time, Features Delivered and Budget.  If the organization fails to deliver the needed Budget, you will need to appropriately change their expectations to align with what can be accomplished within those budgetary limits.

So What Are We Building?

It’s interesting to learn how these types of problems began and they manifested in an organization.  I have shared some insights into the expectation gap and how it happens; now I would like to share a little about how I find these projects started in the first place.

The need for an Intranet Manager happens as a result of many different needs.  Many companies understand they need to share information with others and because of the technologies available, feel an Intranet is the vehicle to do that.

The interesting process I see happen starts something like this; an individual in IT learns of SharePoint’s ability to help them collaborate and manage technology projects.  In most cases, IT is very successful with SharePoint and this is due to many reasons:

1.       The individuals in IT are very technically savvy and understand how to use the features.

2.       These technology individuals understand SharePoint security, because they have an understanding of other security models.

3.       They are using SharePoint for a very specific need and they know how to accomplish it.

There are many other reasons; the bottom-line is “the IT department understands the technology and how to use it to solve their need.”

At some point in the future, someone in the business learns of SharePoint in their environment and requests a site be created for them too.  This type of growth (sprawl) continues in one way, shape or form until one day, the company realizes there are hundreds (or thousands) of sites.  The “help desk” becomes overwhelmed with calls.  Business users cannot find information, search isn’t producing the results they expected, it is unclear where they are to find trustworthy information and so on.

This is not an uncommon evolutionary process for SharePoint.  You may share some similarities in your environment.

Regardless of how you got there, you now have a problem because users in the organization are confused about how to use SharePoint.  Overall trust in the technology begins to crumble because of this confusion.  And, to make matters worse, search doesn’t work at all.

Once your users loose trust in SharePoint, gaining their trust a second time is even more difficult.

So you are at a crossroads; what is it we are trying to build here?  Some individuals in the organization want to use it for technology project collaboration, others wish to use it for business operations documents, and others even want to use it for records management compliance purposes.  And, let’s not forget the unspoken subconscious expectation that search is to produce pristine results!

If this is your target, you have surpassed what most believe an Intranet to be.  Why?  What value did your old Intranet add?  Maybe this doesn’t apply to you, but many people relate the word Intranet with a web site that contained departmental brochure-ware that was out of date, maybe a few important forms and links but not much more.  Meaning they didn’t find it of much value.  Is this really what you have been tasked with?

When I interview customers, I learn something that looks like the following:

1.       All business operations documents are to be managed in SharePoint.

2.       We want to eliminate the file share.

3.       We want to eliminate all documents in Exchange public folders.

4.       We want our users to start sharing information across organizational boundaries.

5.       We want search to produce relevant results, within the context of what I am doing; and I expect those results to be considered published and accurate.

Does this sound about right?

Well… I’ve got more news for you.  This is no longer the old conventional static Intranet.  The business has asked you to implement an Information Asset Management solution that can be used for proactive Decision Support!

The game has changed…  And, you have many new players in that game…

I am happy to report there is a solution!

The key to your success, with regards to delivering this type of solution, is to first architect information in such a manner that it lends itself to producing the expected results.  Secondly, you will find it is important to deliberately design and configure the search technology to support different needs; more on this later.

Setting the Stage

Before we step too deep into the art and science of architecting enterprise information, I want to appropriately set your expectation.  There is a lot, a whole lot, which goes into the creation of an Information Asset Management and Decision Support solution.  It would take many books to cover all of the topics.  In this series of articles I am going to focus strictly on producing relevant search results; if that wasn’t enough!  Much of what I discuss, with regards to search, will apply to the facets of Findability.

By the way; I am writing a SharePoint Planning, Enterprise Information Architecture and Design book…  All in my spare time!  I’ll let you know more as the planned release date comes a bit closer.

It’s All about the Information

The first piece of this puzzle is to thoroughly understand the importance (value) of information in your organization.  As I said above, the value information has in your organization is directly proportionate to the amount of time your organization will spend crafting a solution to produce findable results.  This has been evident with every customer I have worked with.

As an Enterprise Information Architect (EIA), I am constantly working with customers to understand, not only what information they work with, but how it is used and valued.  This process is referred to as understanding information context.  Without context, we have nothing but a document that needs to be stored somewhere.  Without context, we don’t have any basis for understanding its value.

For a specific type of information, once you understand its value, you can determine the amount of time needed to further investigate additional elements.  By this I mean, if certain information has less value to the organization then don’t waste precious time architecting a sophisticated solution for it.  In the Customer Contract example above, the Customer Contract has high-value to the organization; many different individuals use it in many different ways.

To help with the macro-level value assessment of information, I will use another example; an example that compares the value of a Customer Contract versus the Company Lunch Menu.  Both are important to individuals in the organization, but how important?  If these two types of information become unavailable for some period of time, will they both have the same negative impact on the organizations ability to conduct business?  Most likely not; though some may argue that point.  It may be an obvious decision but, I would personally spend much less time assessing the EIA needs of the Company Lunch Menu and much more time on Customer Contracts.  I say this may be obvious, but it may be less obvious with other types of information in your organization.  The value assessment of information is not a black-and-white one; it is one where deliberate thought and strategy needs to be directed.

Now you may be thinking to yourself, “This guy must be crazy!  Does he expect I go through the entire organization and assess every category of information?”  Truth be told, this is exactly what has to be done; if your goals are to create an Information Asset Management and Decision Support solution that produces trustworthy and accurate results.

The breath and depth of your assessment process will directly impact (in a positive or negative way) the quality of outcomes.  And for the purposes of SharePoint Search, it will have a direct impact on the quality of search results.

You still may be thinking; “he is completely out of his mind.”  I have to do this for everything?  It has been my experience that only a fraction of the information in an organization requires deep assessment and resulting design; these numbers can be as low as 20%.  All other information is typically supportive by nature.  An example of this might be the creation of your Employee Handbook.  The creation process can result in hundreds of e-mails, meetings, notes, reviews and approvals; however, the end result is a single document.  In most cases these other supporting documents can be discarded or archived.  And, their value is much less to the day-to-day operations of your business.  In this example, I would be spending more time assessing the resulting Employee Handbook document and much less time with the other supporting documents.  This isn’t to say these supporting documents are unimportant, you may need to archive them for compliance purposes or future review, but they are of less importance than the resulting Employee Handbook document itself.

During the initial assessment of information, I do recommend you place a priority on each category.  Once you have determined the value and priority, you can then begin the assessment process.  When I am working with customers, I follow a methodology that includes 8 elements of EIA assessment.  With these 8 elements, we can obtain most of what we need to create an information map (or taxonomy), design a solution, build it and produce relevant search results.

Below I will provide the 8 elements of EIA assessment but will wait until a future article before I dive too deep into each.

The most important thing to remember is that your information assessment is the beginning of everything; i.e. Information is Everything!

The 8-Elements of EIA Assessment

There are 8 elements to EIA assessment that provide the foundation for implementing Information Asset Management and Decision Support solutions.  Is this something I made up all by myself?  Absolutely not.  This is something I have learned from many individuals and after years of work in this area.  Much of this can be found in various publications available at your local book store and on the Internet.  However, I am going to describe their usage within the context of a SharePoint Information Asset Management and Decision Support solution.

1.       Categories

2.       Understanding

3.       Presentation

4.       Evolution

5.       Knowledge

6.       Responsibility

7.       Process

8.       Metadata

As I said before, I will be covering each of these in a future article.

Conclusion

In this first article it was my goal to help you understand that a thorough understanding of the information in your organization has to come first.  Without being armed with that knowledge, there isn’t any way you can produce a solution that will deliver the expected results.

1.       Clearly understand what the expectations are.

2.       Educate your user-base, helping them understand that producing findable results takes work.

3.       Point everyone down the road of thinking about “Information First.”

Where Else Can You Learn About SharePoint Implementation Strategies, Planning, Architecture and Design?

Free Monthly Webinars

Mixon Consulting, Inc. is delivering free monthly webinars that cover all of these topics.  For more information, visit www.MixonConsulting.com.  If you are unable to find a webinar schedule, then we are in the process of scheduling the next series.  Please feel free to contact Events@MixonConsulting.com for scheduling details.

Mastering SharePoint Community

The Mastering SharePoint Community is available for free, to anyone who is interested in these topics.

http://www.MasteringSharePoint.com