How does Intuit evaluate your performance

This am, someone asked me ‘how do you like working for Intuit?

To be honest, I feel as if I am working for our users, especially those involved in our online community websites:Quicken and QuickBooks 

They are the ones who set our priorities for improving the site. They are the ones who give us feedback on what is working and what is not working. And they are the ones who evaluate our performance.

In fact, when my Intuit manager asks for a list of coworkers he can contact for my 360 yearly review, I will provide him a list of our users as well. Or, just tell him to contact a handful and ask them how they think I am doing.

It drives me crazy that most managers don’t look outside the company for feedback on their team.
Key Plays:

1. User involvement is more than content
2. User involvement is helping set the priorities for product development
3. User involvement means evalutaiting Intuit’s employees’ peformance
4. User involvement can help managers figure out how they can coach their employees.

Social Bookmarks:

In The Trenches

 

Some key plays for building a good community. All of which take place in the trenches.

1. Launch and learn – each team  member must needs contact with users
2. Understand the importance of sharing learnings directly to the developers (better yet, let them hear it from users)
3. User involvement in the development process never stops, never ends (especially as more and more web products are developed)
4. Community is a great place to test new product development processes
5. Change is never constant, it is always happening
6. It is great to collaborate with your heavy users and most valuable customers, but don’t leave out other users, especially since you need to develop to the lowest common denominator, the less experienced and knowledgeable users
7. Verbatims are just as important as ‘numbers and scores’ (Yes, everyone on the team should read them!)
8. Find a users problem and solve it well!
9. Make sure to do a post mortem after your failures and learn from them.
10. Solutions are not always transferable from one product/service to another. (you can not plug in a previous approach to ratings to this group of users

And remember: This all sounds easier than it is. It takes hard work and constant effort.

Social Bookmarks:

Grassroots and Top Down (Part II)

More on selling internally. Continued from previous post:

Remember. All of this takes place in 2004.

Before we launched the QuickBooks Online Community, it was clear that the company had limited experience with online communities or truly online interaction with customers.  We had a small support community. We had played around with live chat as a customer service tool. And we did a few other things in the area of community.

Even though we are located in Silicon Valleyand sit across the street from one of the most innovative technology companies in the world, Google, barely any employees were really active on the community besides shopping at Amazon.

It took some work to find internal thought partners — employees who could coach me on selling ‘Community’ to management. Paul, who headed up the QuickBooks Online product and was probably one of the only other bloggers in the company besides me, provided the guidance I needed.  (There were a few other bloggers, but they were really hard to identify).

So, I started doing Google searches on ‘online communities’ and quickly realized most of the articles were written by researchers and analysts who never built online communities. And as they say ‘it is different when you actually do the work.’ I shouldn’t be so negative because there were some really strong existing communities, such as eBay’s, UrbanBaby, Edmunds, etc.  And we did learn a lot by participating in these communities. We learned. And we shared those learnings with others in the company.  One of the keys to selling to senior management is to make sure nobody at the (grown up) table is surprised by what you show them. In other words, the plan was to make sure that almost of Bennett’s direct reports knew what we were going to share with them before the big presentation.

Secondly, it was important to partner with key internal groups, such as legal, privacy, and finance. Leveraging my working relationship with Paul (he had helped with a number of projects before), we sat down with the General Council and presented our case for why developing an online community would be a game changer. Ironically, we already had an online community up and running, so we had first hand evidence that the company would not fall apart. I  recommend that you not start a community before receiving legal’s approval.

Since part of the plan was to encourage employees to participate in the community, legal was concerned about how we planned to monitor what was said online. That’s when we came up with the idea to provide opt-in training to employees in the company.   In forming my partnerships, I developed some key slogans to reinforce my commitment to get them involved. These included: – Legal is not evil (we even wanted to printed t-shirts with this motto)– Finance can be your friend – Privacy and Security are not trying to restrict you – Product and Customer Service can work together

It was important to get finance on board because it was clear that we would be an expense to the company and any cost savings would have to be identified and (financially) modeled over time. For example, we knew we would eventually try and demonstrate that our users would eventually start going to the community vs. picking up the phone and calling our 800 number. 

Finance also had to be involved because of potential revenue recognition issues. The definition of revenue recognition basically states that if a company discusses (and this includes using the word ‘expects’) a particular feature that is not already in the product, they will have to account for any potential revenue generated by sharing  information on a particular product/service during that period.  For more info on revenue recognition: http://www.sec.gov/interps/account/sab101.htm

Key Play Takeaway: Legal is not evil.

In the middle of things

Received a call from a research person who works at a major consumer goods company. She wanted to know how to wrestle the cattle in her organization and obtain funding for an online community. Her problem was similar to one that I faced earlier in my career when you find yourself in a centralized place in the company and have to pass the tin can around to get funds for your projects. At Intuit, I also face the challenge for the divisions that want me to build a community for them. This time, the project is to build a place where users can talk to other users. Here’s some suggestions

1. Start simple — make your community endeavor a simple project that can be executed easily and that you can show some results sooner than later so that you can lobby for more funds.

2. Pass the tin cup around — get different groups to pitch in $10K here and $10K there. Soon it starts adding up. Or if you are dealing with large groups, tell them you want a percent of the marketing budget. What is 1% of a few million. Or, charge them based on pageviews and any unique functionality they want your online team to build.

3. Co-develop the site — when it comes to the Web, everyone wants to be Picasso (I have never seen so many people — me included — who think they know what they are talking about when it comes to the Internet).

4. Invite them — Invite some of your current customers, whether you have their name in your customer database or know them from another source — invite them to participate in the community

5. Position it well — If you are P&G, are you going to have a P&G site or are you going to break it down and have lots of little brand oriented sites

6. Feel their pain — Every customer experiences some sort of pain or challenge in using your product (except maybe Kleenex’) so start a discussion around that

7. Open or closed — Many package goods companies have closed communities. Not sure why. Ever go to a supermarket on a Saturday afternoon? People are talking about your product. Make a closed community the exception. Allow users to discuss, share and maybe even suggest a better way to do things. Open communities will also grow faster : )

8. Package learnings — Said this before. Sometimes you have to do a little power point action and do a presentation what you learned. (Don’t forget to include VOC)

It is tough to be in the middle — in a cross functional centralized group. Trying to keep everyone happy, engaged, etc.

Rarely are you the approver of a project. So all you can do is make sure you are aligned in what you are trying to accomplish and what you hope to learn from the experience. (Easier said then done)

We are all Ants

Since my earlier days at American Express (AMEX), every company I have worked for has tried to create the uber-customer or knowledge database. AMEX, probably did the best job, though. Considering it was pre-internet when I was there. They recorded the transactions of all their card holders and then used that data to develop co-marketing programs with their merchants and then tailed offers to their members.

At Apple, it was difficult because most transactions went through a third party (Best Buy, Frys and the like). At Borders, only the WaldenBooks side of the house had a membership program to capture useful buyer information.

American Online wasn’t bad at building a knowledge database of sorts because they were a membership / subscription program, so it was easy to know what area of the proprietary services users were surfing in — Even though they had the data they were more into direct mail carpet bombing and sending out as many CDs as possible.

At Intuit, we have a knowledge database, which someday will incorporate most of our user created content. Today, our FAQs written by employees, etc. are separate from those written by the hundreds of thousands of communtiy users. It is worthy trying to create a customer facing, user friendly knowledge database that centralizes all this ifnormation and that can help deflect inbound telephone costs or customers concern about controlling their own destiny, by finding and figuring out the problem on the their own.

So, we spend a lot of energy on creating a clean (good data) and useful (you can find the answer you are looking for) Knowledge Database. Why not let users do this? Maybe you have to reward them, maybe you don’t. If you are facing a similar challenge, a decent short term solution, however, which we are doing on the Quicken Community website is just to ‘grab search strings’ and display the results on a web page. But… if I stop being skeptical, I do see some strong individuals driving for the master database concept. They key is to get your key evangilists — lead users — involved in the project. They are the one who have experienced the pains and gains of financial management. Through some sort of Community Darwinism, these users will float to the top.

Absorb knowledge + Share Knowledge + (have someone else) Absorbe that same knowledge

 

or Learn, Teach and Learn

It is amazing how a corporate philosophy, such as creating a learning culture, applies to the community and of course, to building a Knowledge Database. Imagine what happens when you put lots of users together in a community. Certain thought leaders take the lead. Our job is to make their information available. To also let these thought leaders just emerge from the crowd. They might be a few steps ahead of the rest of the community or the contributors to the knowledge base, but others will follow. That’s why it is also good to recognize these individuals expertise and knowledge — in a particular industry, etc. — and enable them to teach others. After all, they were once the students, whether they read the answer in the book, got it through trial and error learning on the job, or just asked someone else.

I am sharing this because we have learned that it is not really useful to just have a bunch of static FAQs. Especially when dealing with small businesses — where companies are in different industries, users figure out new workarounds, new versions of products are introduced, etc.

Another point is that ratings/usefulness needs to be a living piece of data that evolves and changes over time as more and more users interact with your product. (But similar to the number of people posting on communities, only a small % of folks are really rating posts)

Make sure that the ‘Helpfulness” questions you are asking will provide useful info to your users (of course you). So, you might want to ask more than ‘was this just helpful.’ How about ‘what industry are you in?’ ‘Did you try send another users to read this FAQ, etc.’

Then you can add your ‘helpful’ info to your search results. Where users can search on items that customer voted to be most useful.

In Steven Johnson’s excellent book Emergence, The connected lives of ants, brains, cities and software, we are invited to learn about about the complex engineering feats and research management of building ant colonies. All this is done with any sort of dictatorship — except for the MaMa Ant once in a while. And as the colony is being built ants are dying off and new ones are replacing them. This is analogous to the community and to the contributors of the knowledgebase

The behavior of one ant is based on the behavior of all the other ants, which is based on random events. And No one will be participating for ever. And when they leave (hopefully not due to death), others will replace them and follow the written and sometimes unwritten laws.

Johnson points out that if you are building a system frm the ground level — like a KB system or a community, there are learnings at the global/macro level, but also at the local level. I will touch upon them below and take a little bit of creative license in explaining them.

 Key  Plays

1. More is different: You need to have critical mass in order to study the colony, but more important to make sure you are getting proper representation — which will highlight the most common issues…..I guess this is similar to making sure you have both a statically significant sample and that the more data you have, the easier it will be for themes to float to the top.

2. Ignorance is useful: Build a simple system — sometimes we try and over-engineer our databases — design for dummies — or the lowest common denominator, your lease experienced user.

3. Encourage Random Encounters: Let the users connect the dots between the type of product they are using and how they are using it. We learn so much from our community about how they use our products — A great example is Workarounds. So many of them are discovered and shared out of the blue

4. Look for Patterns in the signs: You will see some common themes float to the top.

5. Pay Attention to Your Neighbors: “Local information can lead to global wisdom’ — We learn so much by looking beyond our own borders — our own websites — and seeing what other small business owners are saying on the web.

Your Knowledge Database should be thought of as self-organizing and living organisms. Don’t try and control it…(how many times have you heard that about communities? The difference here is that I am also saying it about Knowledge Databases)

Start simple and don’t try to centralize all your existing data. Let your users be like ants and work on it together, whether it is intentional or unintentional. All you are doing is providing the infrastructure for them to play and stay a while. I might be alone here — and be perceived as someone who is against extensive planning and design. Lets just say that while I believe in some design of the KB, the power will be to let it evolve on its own. How many of you are struggling to build your own super customer database or knowledge database, eh?

(For a good enterview with Steven Johnson)

More on ROI — and reporting

6 months after we launched the community site, I was provided assistance and loaned 30 employees from other parts of the company. The goal was to help jump-start the online community and also put some systems and processes in place so that it could easily scale.

Unfortunately, I think too many cooks were in the kitchen, and that we got a lot more done — got a lot more traction later on when I was able to hire 10 employees. One of the ‘loaners’ was a Six Sigma Black Belt. While I am a big fan of Six Sigman, I think we made the mistake of over-engineering our dashboard when he came on board. Below is a snap shot of all the items we wanted to track. 18 months later, we do track almost all of them.

The problem is that if you can’t your point across with 10 different numbers (or even less) than you haven’t really defined the important drivers of your business. I guess one could argue that it is important for me, the general manager to have my pulse on all aspects of the business. I believe, however, that it is important for a community team to define the ten most important items to track and establish a process where learnings can be shared with other parts of the company and that users can either get their questions answered or obtain valuable and useful information to help them in the personal and / or professional lives. Below is a list of some different types of users:

1. Streakers: Seriously, there are many users who come on the site and just go to 1 page and never come back

2. Unregistered Lurkers

3. Registered lurkers: Users who register at the site, but never post

4. Users with X number of posts within different time frames (weeks, months, etc.)

5. In-active registered users: Registered users who hanve not posted in a while

6 High Resolution/Answer Person (Answering 50+ or more posts)

The original problem with this is that someday 50+ will have to change plus some users do not just answer posts. Some just reply and then it is up to the other users to determine if the posts are useful or not

7. Users who ask and answer multiple questions

8. Registered and Active

– Active was defined based on activity, which I think is a good proxy. The activities included doing a search, posting, subscribing, etc.

9. AllStars or Key Answer People:

– This could be based on X number of posts or replies OR someone who the community or a moderator has annointed as a person who adds tremendous value to the community

10….. WELL, I know that I am leaving out several. What group(s) do you think I should add?

Social Bookmarks:

QA and Web 2.0

This past week, we relaunched the Quicken Online Community website. In preparing for the launch, we worked very closely with our users to figure out ‘what we should solve for’ with the new site, how they want to be ‘kept in the loop’ on the launch (yes, you should tell them them when the old site is being taken down), when the new site will launch, etc. We also tried to share with them our road map about what new features and functionality will be on the site.

So, what was our internal process? Our team put together a simple database — we used our own product, QuickBase — to track items to be tested and reviewed, who on the team would be assigned to QA these items, and where each ‘part’ was in the QA process (reviewed once, reviewed twice, changed by an engineer, etc.).

So everyone on the team touched (QA’d) their areas of expertise. The moderators tested out our forums. I pounded on the Expert Locator, etc.

Key Plays

1. Have everyone on the team and beyond (coworkers, users, etc.) QA the site
2. Count on doing sme clean up after the launch, especially because your have a forum for users to easily tell you what is not working
3. Have a dedicated place on the new site to keep users in the loop on what is going — updates, changes, bug fixes, etc.
4. Have a dedicated place for users to log issues — they can be your best QA testers (could be the same as item #3)
5. Build in some human redudancy and test certain items multiple times by multiple people
6. Don’t take away any existing functionality users are already comfortable with… but this can be a challenge
   • because the loudest users might not be using the most popular tools
   • because there are few good community platforms on the market and no matter who you go with, you will have to make tradeoff

So, we can’t reall do any fancy Web 2.0 without thinking seriously about QA (Quicken Assessements)!

Social Bookmarks:

My first day at school (ah..I mean Intuit)

                          I will never forget my first day at Intuit. After sitting in a three hour training meeting (which we call ‘onboarding’) about the company’s benefits, leadership practices and products, my manager brought me to a meeting with Scott Cook. We shared with him some ‘data on how the site was doing and how we planned to change the site.’ His initial response was ‘what are the website’s users saying.’ Instead of just looking at the numbers, he wanted verbatims. He wanted VOC – he wanted to know the ‘voice(s) of the customer.’ I have to admit that this was very different from what I had experienced at some of my other companies, where management just wanted to ‘see the numbers.’ Since then, I have noticed that our product managers relay on what customers are actually saying and what types of words they are using.’  

Now the Online Team reports not only on the numbers, such as unique visitors, posts, posts replied to by forum, but also share the actual verbatims with our product management team….not only from Intuit sponsored online community websites but also third party websites for information about our products.

Since day 1, Scott’s question about voice of the customer ‘VOC’ has lingered in the back of mind.

By the way…. I love going to school and am always in learning mode.

Thanks Scott!