The Do Phase
Step 4: Collect and analyze the data.
"If
you Don't measure it, you don't manage it." (Juran, 1989)
Input to Step 4: The input
to Step 4 is the output from Step 3.
- A list of possible
benchmarking partners from research
sources (approximately 15). - A blind list of potential
best practices (approximately 5-15).
- A list of the final
partners selected (approximately 1-5).
- Team copies of the
Benchmarking Code of Conduct.
A.
Determine the data collection plan and method.
Now
the BMK Team needs to determine a plan and agree on a method to collect data
about the benchmarking process and any performance measures to be used for
comparison(s), from within their own organization as well as from their
benchmarking partner(s). The goal is to collect valid, reliable, objective performance
data on the internal process first. Examples adapted from the American
Productivity and
·
Productivity, by transactions per unit.
·
Accuracy, by the error rates.
·
Responsiveness, by the time intervals.
·
Speed, by the cycle time.
·
Product stability, by the engineering change orders per month.
·
Process financial contribution, by the value-to-cost ratio.
·
Product availability, by fill rate.
·
Product quality, by first-pass yield.
·
Capacity, by volume managed.
·
Service, by the on-time delivery.
In
developing a data collection plan, the BMK Team should answer the following
questions:
- What data will give us
what we need to know to compare our process with the best-in-class
(inventory control, recruitment, procurement, education and training,
marketing, etc.) process?
- What kind of
information/measurement is necessary (accuracy, quality, customer
satisfaction, speed, dependability, etc.)?
- What data exist on the
internal process?
- How should the BMK Team
collect the data?
- Who on the BMK Team
will collect the data?
- How will the BMK Team
check its results?
- How much time will be
needed to collect the data?
- How will the data be
consolidated and analyzed?
- How will we evaluate
the satisfaction of the customer(s) of the process?
- What method(s) should
be used to collect data from partner(s)?
- hard copy correspondence
(mail/e-mail/fax)?
- z telephone interviews?
- z publications/other
media?
- z library and database
research?
- z personal
interviews/meetings?
Methods of Data Collection
One
(or more) of the following methods can be used to collect the data. Following
are some guidelines for each method and some advantages and disadvantages of
each.
Correspondence. Using hard copy correspondence such as the U.S. Mail service, electronic
mail, or fax to collect data is an inexpensive, easy, and time-efficient way to
gather this information. However, correspondence limits the ability to probe,
and may require follow-on questions. Be aware that some organizations may not
give .answering the mail. a high priority.
Telephone. A telephone call is easy to plan and conduct. It facilitates contact with
a large number of partners and can be relatively inexpensive. It provides a
direct, personal contact with your partner(s). It also provides the ability to
get a better sense of the organization and the individual with whom you are
dealing. A common problem with telephoning, however, is that it can be difficult
to .connect. with the person you wish to speak to
(a.k.a. phone tag). In addition, a .cold call. can be time consuming and frustrating for all parties. It is
recommended that you send a read-ahead package to prepare your partner(s).
Include a suggested date and time and an estimate of the time required for the
call to increase your chances of finding your point of contact available and
informed. Contact a specific individual and maintain a good working
relationship with this person. Explain again who you are and why you are
calling. Mention any referrals. Exchange information where appropriate.
Establish a follow-up session where necessary.
Publications. Publications and other
forms of media, including World Wide Web sites, hold vast amounts of useful information,
provide many opportunities to advertise for a partner, and often provide clues
as to who may be considered the best-in-class. Magazines and journals often
have articles on the pacesetters in a particular process. An ad in the
newspaper or a trade paper can be minimally expensive and might solicit some
surprising partner(s).
Research. By this point, the BMK Team
has already done research to identify partner(s) via the library and database
research. The BMK Team members can sift through this information to see what
may already be contained and useful for this particular step of the process.
Interviews. Face-to-face contacts
through personal interviews and meetings represent a powerful methodology.
Conferences, meetings, training sessions, etc., provide informal opportunities
to talk to others about what they do and how they do it. But this can become a
resource-intensive method of gathering information from possible benchmarking
partner(s), and, most importantly, it doesn't guarantee that you will find the
recognized world-class organizations. It can also become awkward if the
partnership doesn't work out as anticipated.
Site
visit. It is possible to have a
successful benchmarking study without a site visit. Sometimes through the use
of technology, such as teleconferences and a groupware system, the information
you need can be acquired at low cost. However, if it is necessary to go to a
partner's location, here are some guidelines for the visit:
- Make contact with the
appropriate person to provide the proper authority for the visit.
- State the purpose of
the visit.
- Verify the suitability
of the site.
- Offer a reciprocal
visit.
- Identify mutually
agreeable date(s) with start and end times.
- Select a limited number
of benchmarking team members to visit (2 to 5).
- Send a letter to
confirm the visit and include:
- the date(s)
- those who will be
making the site visit
- those who you propose
to visit at the partner's site
- a proposed agenda]
- a proposed time frame
for visit
- a flowchart and/or
explanation of the process you plan to benchmark
- the data collection
process you plan to use
- any ground rules.
- Be sure to check on any
security check-through procedures.
- Get clear directions on
how to get there and where to park.
A Word of Advice: Don't rush off and do the site visit before the benchmarking team is adequately
prepared. You want to be sure to use your time (as well as your partner's time)
during the site visit effectively and efficiently. Send the right people and be
prepared to provide business cards. Listen. Stay focused. Test for a common
understanding among all internal and external parties throughout the site
visit. Debrief as soon as possible; always debrief one site before you go to
another. Neutralize emotions and be objective. You may find some great
personalities at some great locations, but how great is their actual process?
Survey. Many organizations use survey instruments or a questionnaire to help
focus the effort and standardize the information collected from various
partners. A survey should consist of open-ended questions developed by the BMK
Team. The questionnaire should be limited to no more than 15 questions that
would take no more than one hour to answer.
Regardless
of how contacts with potential partners are made, the same questions should be
asked of each partner. This will enable the team to have like-responses for
better comparisons. You should be able to answer the same survey questions for
your own process. The answers to the survey reveal a lot about an organization's
understanding of the benchmarking process as well as about their own business
process.
B.
Collect and rank the data.
Now
the team can actually begin the comparison of its business process against
those of a world-class organization. Designated BMK Team members should contact
the partner(s) and collect the data based on the plan and methodology developed
by the team. After the data are collected, each partner is ranked in
performance measurement order. This identifies where your partner's performance
is significantly above and below your current performance level.
After
collecting the data about the process from the benchmarking partner(s),
establish your own ranking and any performance gaps. This provides the basis
for performance goal-setting (Step 6). Having a measurement system in place
allows you to measure your progress toward the goals.
Partner(s)
should be blindly ranked; that is the actual names of the organizations should
be replaced with symbols such as Company #1, Company #2, and Company #3, etc.,
or Organization A, Organization B, Organization C, etc. The example that
follows is a simple matrix showing generic performance measures and where each
organization, as well as your own, ranks. It uses a 1 (best) to 5 (worst)
numbering system. With this method, the team can quickly see which
organizations are the best of breed. In this case, the lower
the number the better.
Ranking the performance measures of
a pharmacy
A Word of Advice: The goal in ranking
performance measures is to seek direction and categorize partners. Don't spend
an inordinate amount of time splitting hairs between which organizations should
be rated number 4 or number 5.
Look
at the gaps in rankings and try to determine some of the reasons for the gaps.
Project any future competitive gaps you may be aware of due to things such as
evolving technologies. Camp stresses to look for balance in measures, not just
cost (Camp, 1996). Things like quality, accuracy, delivery time, asset
utilization, and the level of customer satisfaction in products should also be
measured and ranked. Summarize the findings for the benchmarking report (Step
5).
C. Train
the BMK Team with just-in-time skills/tools as needed.
When
you assign the roles and responsibilities that each BMK Team member will have
in researching, collecting, analyzing, and documenting the internal and
external benchmarking data, consider if they will require specific training to
participate. For example, training in: Many quality tools are
available to assist the BMK Team in data collection and analysis. Those
in your organization who have successfully completed the DON TQL Team Skills
and Concepts course and/or the Systems Approach to Process Improvement course
are trained in the use of these tools. Contact your TQL coordinator or
specialist as needed. Some tools commonly used in benchmarking studies are:
·
survey instrument development.
·
use of databases/technology/World Wide Web sites.
·
matrix development.
·
data collection methods.
·
statistical analysis.
- fishbone diagram.
- matrix.
- pareto analysis.
- histogram.
- affinity diagram.
- scatter diagram.
- run chart.
- storyboarding.
- integrated computer-aided
manufacturing definition language (IDEF) modeling.
- cost/benefit analysis.
Output from Step 4: The
output from Step 4 is the input for Step 5.
- A data collection plan
and method.
- Quantitative (just the
numbers) and qualitative (what the numbers mean) performance measures of
the process.
- A blind list of
partners in order of where they rank in each performance measure.
Quality
Advisor's Checklist
Before
moving to the next step, the quality advisor should review the following
checklist:
·
Will the data collection plan and method provide valid, reliable, and
objective performance information on the process?
·
Are all BMK Team members collecting data in a similar fashion?
·
Is a site visit necessary?
·
Were potential partners ranked blindly?
·
Does the BMK Team have all the tools and skills necessary to collect and
analyze the data?
- Did the BMK Team
document, document, and document some more? z Is
there software available that could simplify the data collection and
analysis? z Is there information that can be
collected through secondary research without wasting resources?
- Is there information that can be
collected through secondary research without wasting resources?
Disclaimer: This material is for training purposes only. Its purpose is to inform employers of best practices in occupational safety and health and general OSHA compliance requirements. This material is not, in any way, a substitute for any provision of the Occupational Safety and Health Act of 1970 or any standards issued by OSHA.