SlideShare une entreprise Scribd logo
1  sur  23
Télécharger pour lire hors ligne
HARTSFIELD-JACKSON ATLANTA INTERNATIONAL AIRPORT - CITY OF ATLANTA
Aviation Procurement Unit
RFPEvaluation
TrainingManual
A V I A T I O N P R O C U R E M E N T U N I T
RFP Evaluation Training Manual
© 2009 City of Atlanta. Aviation Procurement Unit
6000 North Terminal Parkway • Atrium Suite 4000
Atlanta, GA 30320
Phone 404.530.5408 • Fax 404.905.1589
Table of Content
Introduction................................................................................ 1
Definition ................................................................................... 1
The RFP as a Solution .............................................................. 2
When to Use an RFP Approach?.................................................... 2
What is the Best Way to Handle Criteria for RFP Evaluations? ...... 3
How Do We Develop an Evaluation Strategy?................................ 3
What are Examples of Standard Criteria in a Technical Proposal?. 4
Overview of the Evaluation Process.......................................... 6
Standards of Conduct While Serving as an Evaluator..................... 6
What is Expected from Evaluation Committee Members? .............. 7
Oral Interviews ................................................................................ 7
Reference Checks .......................................................................... 8
Ensure Security and Confidentiality ................................................ 9
Approach to Evaluating Proposals .......................................... 10
Evaluation Scoring Guidelines ...................................................... 10
When is a Proposal Non-Responsive?.......................................... 11
Approach to Evaluating Technical Proposals................................ 11
Approach to Evaluating Cost Proposals........................................ 13
Consensus Evaluation and Scoring ........................................ 15
Benefits of Consensus Evaluation and Scoring ............................ 15
Preparing for the Consensus Evaluation Session......................... 15
Steps to the Consensus Scoring Evaluation Session ................... 16
Role of the Facilitator .................................................................... 16
Documentation Steps for Consensus RFP Evaluations ................ 17
Helpful Hints and Best Practices............................................. 18
Next Steps: The Recommendation ......................................... 18
Frequently Asked Questions ................................................... 19
T H E R F P A S A S O L U T I O N
1
Introduction
A Best Value Award is one which optimizes quality,
performance, cost, time, and efficiency in a fair
manner to the contractor community within the contract.
ids and quotes cannot always address the needs of the Department of
Aviation (DOA). Generic specifications may not be available or difficult or
impossible to draft and conventional evaluation for award based on lowest
cost bid may not get the product or service required. Many high tech
products and complex services cannot be obtained by conventional bidding. The
Request for Proposal (RFP) is a solicitation used for situations like these.
Definition
The RFP is a formal competitive sealed bid process. The RFP
outlines the requirements of the Business Unit by describing the
purpose, scope, description, minimum requirements or expectations, qualifications or
capability of the proposers, evaluation criteria, and other requirements. In the RFP
response, the vendor offers a solution for the particular need described in the RFP.
The RFP is evaluated according to predetermined weighted standards.
After completion of the RFP process negotiations may, in some circumstances, be
utilized to secure more advantageous terms or reduced cost.
Volume
1
B
R F P E V A L U A T I O N T R A I N I N G M A N U A L
The RFP as a Solution
When to Use an RFP Approach?
A successful Request for Proposals (RFP) requires much planning.
No two RFP’s are alike and preparing one can be a difficult task.
However, a well-written RFP can alleviate many problems. Business
Units are encouraged to involve the Aviation Procurement Unit
(APU) early in the development of the RFP. APU can provide
assistance and valuable advice.
The following are typical instances when an RFP is the best approach;
When the use of competitive sealed bidding is either not practible or not
advantageous to the DOA.
When we are not sure of the best solution and multiple options exist.
When performance-based incentive contracts are used.
When complex factors in addition to price are very important.
When there are several complex requirements.
When it is a new program/ project that has not been tested.
When it is desirable to find out if there are alternative options to what is
normally performed.
A RFP is one of several procurement tools. APU will work with the Business Unit
Project Manager to determine whether it is the best tool for the goods or services that
are needed. The Chief Procurement Officer (CPO), upon receiving recommendation
from APU and the Contracting Officer (CO), will ultimately determine if the RFP
process is appropriate.
NOTE:
The bid process is the default solicitation option.
T H E R F P A S A S O L U T I O N
3
What is the Best Way to Handle Criteria for RFP Evaluations?
It is important to identify all evaluation criteria and their relative importance, including
price, early in the RFP development stages. These criteria will be the only way to
properly evaluate the proposals and assure that the awarded proposals meets all the
requirements of the solicitation. Criteria not specified in the RFP cannot be used for
evaluating the proposals.
Begin by making a detailed list of the most important aspects of the services or goods
required, including cost. Each item on your list is a potential evaluation criterion.
Arrange the list in the sequence of what is most important.
Next, assign a point factor to each criterion based on its relative importance. The most
important items should be evaluated heavier and have more points available. Points
assigned to each criterion are included in the RFP. Including points makes offerors
aware of which items are relatively more important than others can influence an
offeror in the preparation of their RFP response.
All Evaluation Criteria MUST be established prior to the
issuance of the RFP.
Criteria should be clearly linked to desirable outcomes.
Be Clear and Concise in language to avoid confusion.
Evaluation Information MUST be documented, dated, and secured
throughout the evaluation process.
How Do We Develop an Evaluation Strategy?
The following should be considered in developing your evaluation strategy.
1. Identify (eliminating) Minimum Qualifications
These are requirements that a vendor must meet in order to accomplish the work
outlined in the RFP. They may include such things as proper licensing or
accreditation and special insurance or bonding. They are evaluated on a strictly
pass-or-fail basis. Make a list of the things that will be absolutely required for a
successful offeror to have in order to enter into a contract. Do not include
“desired” things or items that could be obtained by an offeror at a later time prior
to contract award. Generally, if an offeror fails on any portion of the mandatory
requirements, their proposal will be rejected, so put careful thought to this step.
2. Define Criteria
Develop Statement of Intent or Purpose and Scope of Work - The
Statement of Intent or Purpose is a description of the general type of
T H E R F P A S A S O L U T I O N
4
service or goods required. The Scope of Work is a general summary of the
work to be performed by a contractor. Developing these brief descriptions
first will assist you to begin organizing your thoughts and help you decide
on proper evaluation factors.
Decide importance of Oral Interviews/ Presentations - To properly
evaluate proposals, oral presentations may be scheduled to answer
questions for evaluation committee members. After consultation with the
CO, only those firms that are potentially acceptable are invited to
participate in oral presentations. The offeror’s original proposal cannot be
changed in any aspect at the oral presentation. The oral presentation is
only to allow offerors to clarify portions of their proposal.
Incorporate Reference Checks into Industry Experience - The
purpose of contacting references is to verify the corporate capabilities and
prior performance of the proponent and the qualifications of proposed
project personnel. Reference checks are made by telephone. The APU
Contracting Officer designated to assist the Evaluation Team will contact
the references identified by the proponent. The results of the reference
checks are compiled and provided to each evaluator to assist in scoring
specific criteria. All reference questions must be relevant to the scope. The
same questions must be asked to all references for all proponents and
should be structured to assist the evaluators as they review information
from each reference and to support follow-up questioning on selected
reference responses.
Decide importance of Site Visits –The purpose of the site visit is to
acquaint the vendors with the conditions under which the work must be
performed.
3. How important is Cost?: Cost to Technical Score Ratio;
4. Determine distribution of points among each criterion.
What are Examples of Standard Criteria in a Technical
Proposal?
The following are examples of standard criteria in a Technical Proposal.
I. Corporate Experience:
What are the proponents’ past performance and experience in similar
types of contracts?
What is the experience level of the essential subcontractors?
T H E R F P A S A S O L U T I O N
5
II. Project Organization and Staffing:
What is the staffing approach?
What are the qualifications and past performance of the personnel?
III. Implementation Approach:
What is the adequacy of their work breakdown structure/ project
management?
Is it Comprehensive? Reasonable?
Proposed Deliverables?
Preparation Plan?
IV. Operation/ Management Approach:
Plan for Regular/ Daily Operations?
Approach to Quality Control and Assurance?
What are the Commitments to Performance Standards?
What are the Plans for Maintenance?
What are the Contingency Procedures in the event of an emergency?
V. Corporate and General Capacity:
What is the overall capacity of the organization?
Do they have a plan to sufficiently staff the contract project?
How long has the company been operating?
What is the turnover ratio?
R F P E V A L U A T I O N T R A I N I N G M A N U A L
Overview of the Evaluation Process
Evaluation of the RFP responses is best accomplished in a team effort. The evaluation
committee is comprised of the evaluation team, the APU Contracting Officer and the
Department of Procurement (DOP) Contracting Officer as the Facilitator.
NOTE:
APU facilitates and coordinates processes between the Department of
Procurement and evaluators.
Rules and regulations that are applicable to the evaluation process must be followed
throughout the entire process. The following is a brief overview of the evaluation
process:
Evaluation committee members independently review proposals based upon
the specific evaluation criteria.
Technical and cost scores are assigned applying the designated evaluation
criteria.
Oral interviews are conducted, if necessary.
Negotiations are conducted with the winning proponent.
Standards of Conduct While Serving as an Evaluator
The evaluation team should be made up of individuals with varied talents and expertise
to assure impartiality. Team members need to be aware of the possibility of an
extended time commitment before agreeing to be a participant.
The following standards of conduct must be adhered to while serving as an evaluator:
No Conflicts of Interest
No solicitation or acceptance of gifts or anything of value
Integrity and Honesty during scoring and communications
Proper handling and discretion of all information; No Disclosure during
process
Practice Ethical Behavior
R F P E V A L U A T I O N T R A I N I N G M A N U A L
7
What is Expected from Evaluation Committee Members?
The evaluation committee members play a significant role in the decision to select a
contractor, which results in the award of a contract. Accordingly, it is imperative that
everyone understands what makes a good evaluation committee member. The
following are standards established by DOP and APU;
1. The member has knowledge or experience regarding the RFP subject matter.
2. The member has sufficient time to dedicate in order to effectively evaluate the
proposals.
Such time is dedicated to reading all proposal related materials
extensively.
Mandatory attendance to all evaluation-related meetings: trainings,
evaluation sessions, oral interviews, debriefings, criteria planning.
3. Conduct an independent review.
Identify potential deficiencies in a proposal that could result in a
determination of non-responsiveness or non-responsibility.
Limit communication regarding the RFP project to those DOA and
DOP staff who are directly assigned to the project.
4. Refrain from any communications with potential vendors/ contractors.
5. Determine whether or not oral interviews will be necessary.
Oral interviews should follow group discussions
Oral interviews should ALWAYS precede scoring.
Oral Interviews
In complicated services, it may be beneficial to require offerors to make an oral
presentation. Discussions may be conducted with responsible and responsive offeror’s
who submit proposals determined to be reasonably susceptible of being selected for
award.
The purposes of Oral Interviews are as follows:
Allows proponents to clarify on areas that may be unclear in their proposals.
Possibly reduces the likelihood of misunderstandings.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
8
The evaluation committee members determine whether or not oral interviews will be
necessary. Oral interviews should follow group discussion, and should ALWAYS
precede scoring.
Reference Checks
Reference Checks will be conducted with prior customers of the
proponent. Reference Checks may also include references of key
personnel. A standard form is used to collect responses. However,
additional subject matter questions may be included when requested.
When additional reference questions are to be included as part of the
evaluation, the following information is required:
List of questions for references.
Instructions to be given to the reference including a numerical scale to be used
in rating the offeror.
References should be contacted only once. The APU Contracting Officer will conduct
the reference check, which may include a conference call with all members of the
evaluation team present when there are subject matter specific questions. A Reference
Analysis will be provided to the Evaluation Team prior to the collaborative scoring and
evaluation session.
The Reference Verification Form includes factors/ratings for technical performance,
management performance, and customer satisfaction. The references will be asked to
rate their level of satisfaction from 1-5.
Examples of Questions include:
Were the major tasks/milestones/ deliverables on schedule?
Were they able to identify and solve problems expeditiously?
What was the effectiveness and reliability of key personnel?
Were they able to recruit and maintain qualified personnel?
Were they able to effectively manage essential subcontractors?
What was the overall performance in planning, scheduling and
monitoring?
How would you rate their overall technical performance?
How would you rate their overall management performance?
R F P E V A L U A T I O N T R A I N I N G M A N U A L
9
How would you rate their ability to be cooperative, business-like and
concerned with the interests of the customer?
Would you do business with this vendor/ contractor again?
How would you rate your level of satisfaction from 1-5?
Ensure Security and Confidentiality
Throughout all phases of the evaluation, the confidentiality and
security of proposals and the scoring process must be maintained.
To ensure confidentiality and security, evaluation sessions will be
closed to the public and staff who are not supporting the Evaluation
Team.
The evaluators shall not discuss the contents of proposals or the procurement activities
with any persons outside of the evaluation team or Procurement staff. All evaluators
and all other staff involved in the evaluation effort must strictly adhere to the following
requirements.
Evaluators must keep proposals in a secure place.
Evaluators are not permitted to discuss the procurement or evaluation process
with any persons outside of the evaluation team or procurement staff.
Evaluators shall not communicate the scoring outcomes or content of
proposals and shall not disclose the status of any proposal.
Cost components shall be evaluated upon the completion of the scoring of the
technical components.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
10
Approach to Evaluating Proposals
All proposals shall be evaluated in accordance with the City of Atlanta’s Procurement
Code and the criteria specified in the RFP on the Percentage Evaluation Form and
considering the information required to be submitted in each Proposal.
The evaluators will independently read and assess compliance
with the criteria identified in the RFP to assess the
completeness, quality, and desirability of proponent responses.
All members of the Evaluation Team will document their
findings for evaluation criteria within each of these areas.
Each criterion is independently reviewed by evaluators, documenting the pros and
cons of each. Review reflects individual, independent evaluations of a proposal and
response to criteria. Criteria include questions or items for consideration for evaluators
to use as a guide in discussion during the collaborative scoring and evaluation session.
The Evaluation Team should not score the criteria during their independent evaluation.
Evaluation Scoring Guidelines
Each technical proposal will be evaluated against a set of pre-determined criteria to
assess the degree to which it meets that criterion. Compliance with requirements will
be assessed as a point score on a scale from 1 to 10. Technical criteria will be scored as
shown below:
8-10: Outstanding: Proponent’s proposal exceeds expectation and demonstrates
an excellent ability to reach the goals and objectives of the procurement.
5-7: Highly Effective: Proponent’s proposal demonstrates a good ability to reach
the goals and objectives of the procurement.
3-4: Effective: Proponent’s proposal demonstrates a fair ability to reach the goals
and objectives of the procurement.
1-2: Marginally Effective: Proponent’s proposal demonstrates a minimal ability
to reach the goals and objectives of the procurement.
Scores will be adjusted to reflect the weight assigned to each criterion. Each criterion
must be reviewed according to the requested items of the RFP and scored based on
the scoring definitions above. Only the Criteria in the Percentage Evaluation Form will
receive scores.
All notes taken
regarding any
proposal are
subject to the
Open Records Act.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
11
Note
Any Proponents’ Proposal that does not meet the minimum
requirements of the RFP should be identified as this may deem them
non-responsive.
When is a Proposal Non-Responsive?
Only the Department of Procurement has the authority to determine whether a
proposal is non-responsive or non-responsible. A Proposal will be determined to be
non-responsive when the proposer is not responsive to the mandatory requirements
clearly established within the RFP.
If an evaluator at any time has a concern with the proponents’
proposal meeting the stated requirements, they should
immediately notify the Facilitator and Procurement staff.
Approach to Evaluating Technical Proposals
The purpose of this phase is to measure the individual merits of the technical
components of the proposal against pre-established criteria. Members of the
Evaluation Team will review all Technical Proposals that pass the Mandatory Technical
Requirements.
Evaluators are instructed to read through the proposal Executive Summary (if present)
before beginning to evaluate and score detailed criteria. The material in the proposal
introduction should provide all evaluators with a broad understanding of the entire
proposal. In assessing individual responses within a section, evaluators may elect to
review related topics within other sections of the proposal. Evaluators are restricted to
evaluating information contained within the proposal. Information not part of the
proposal may not be considered.
Evaluators will proceed according to the following steps to evaluate responses to each
criterion:
1. Review the appropriate section of the RFP.
2. Locate the section(s) of the proposal where the criterion is addressed.
3. For each criterion, note the RFP sections referenced in this document.
4. Review and evaluate section(s) of the proposal.
Evaluators should
promptly notify the
CO if there is a
problem with the
Proposal.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
12
5. For each criterion, evaluate and assess the criterion based on the
proponent’s overall response to the requirements indicated for the
criterion (see step 8). Some of the criteria have additional questions that
may be considered in the evaluation.
6. Evaluate how well the proponent’s responses in the referenced sections
correlate with other pertinent sections of the RFP and the overall
approach taken to address the technical components of the proposal.
7. Evaluate the criterion based on all information available that pertains to it
directly or indirectly, including reference checks.
8. Develop a list of pros and cons for the criterion based on the evaluation of
the proponent’s capability to meet that criterion.
Evaluators are encouraged to request technical support from the CO or from the
Department of Procurement in preparation for criteria documentation. Training will be
provided at the inception of each Evaluation Committee, and prior to proposal
distribution.
WhatdoesTechnicalScoringLookLike?
Weight Criteria Company 1 Company 2 Company 3
20% Organization/Resumes of
Key Personnel
10 3 1
30% Overall Experience,
Qualifications and
Performance of Previous
Similar Projects
9 7 4
20% Strategic Approach To
Increasing Federal
Assistance to the City
through Government
Agencies and Congress
8 6 3
R F P E V A L U A T I O N T R A I N I N G M A N U A L
13
Approach to Evaluating Cost Proposals
After technical aspects of proposal responses are initially reviewed, then cost proposals
are reviewed.
Cost proposals may reflect price escalation, deceleration due to performance,
time, CPI, etc. may be applied.
Cost Evaluation must be independent of the Technical Evaluation.
A Clear Formula must be used to Calculate Cost Scores.
Scores should be directly linked solely to costs proposed on a formulaic basis.
The cost proposal/ financial offer must be taken at face value when the cost
proposal is evaluated.
Note
Scores are documented only in the Evaluation and Collaborative
Scoring Session by the Facilitator/Procurement Staff.
Traditionally, the Lowest Fee Offer receives the maximum points possible (10). In the
event that we are soliciting a revenue generating proposal, the highest offer would
receive the maximum points. Points are distributed based upon percentile in which the
fee offer is found in relationship to the lowest offeror.
Example Cost Offer Points
Cost Proposal 1 $780k 10 Lowest Offer
Cost Proposal 2 $1.3 mil 6 780k/1.3mil = .6
Cost Proposal 3 $1.8 mil 4.3 780k/1.8 mil = .43
Formula Used: Lowest Cost Proposal Offer/ Cost Proposal Offer
at Hand = Percentage Relationship to Lowest Offer
R F P E V A L U A T I O N T R A I N I N G M A N U A L
14
WhatAreSomeExamplesof CostProposals?
The following are examples of what the cost proposals may read like for the types of
RFP’s and resulting contracts:
Type of Contract Suggested wording… Scenario Example
Performance Based Please propose Base Costs and
Incentives associated with
performance beyond
mandatory requirements…
Beyond 90% Customer
Satisfaction Scores
consecutively for one
year will lead to a 3%
bonus (e.g.., janitorial)
Cost Savings Please propose Base Costs and
Incentives associated with cost
savings beyond mandatory
requirements…
Company X will
receive 10% of all cost
savings beyond
performance guarantee
(e.g.., workers’
compensation)
R F P E V A L U A T I O N T R A I N I N G M A N U A L
15
Consensus Evaluation and Scoring
Consensus Scoring consists of the Entire Evaluation Committee
participating in Joint Discussion, Assessment, and Scoring of
Technical and Cost Criteria in order to develop one unified score for
each criterion.
The evaluators are allotted five (5) days to independently review the
proponents’ proposals. Next, the evaluation committee will meet to evaluate the
proposals, discussing the pros and cons of each proposal as well as clarifying their
assessment.
Benefits of Consensus Evaluation and Scoring
While the benefits of consensus evaluation and scoring are vast, the following portrays
the immediate results;
Team Discussion leads to greater Sharing of Each Other’s Expertise in Cross-
Functional Projects, which are quite common in DOA RFPs
Generates Clarification where Misunderstandings May Exist
Ensures that All evaluators are truly Engaged in the Evaluation process
Encourages deeper Probing, Questioning, Discussion, and Analysis of the
content of Proposals in response to the Evaluation Criteria
A Unified Understanding of Responses to All Proposals is reached
Eliminates discrepancies among scores
Overall, the result is a more Comprehensive and Accurate Assessment of the
proposals
Preparing for the Consensus Evaluation Session
In preparing for the consensus evaluation session, evaluation team members should
review thoroughly and comprehensively all proposal responses, repeating review of
responses where necessary.
Keep in Mind How the Information Relates Specifically to the stated Criteria
in the RFP.
Identify All Points of Information in the Responses that Need Clarification
Identify the Pros and Benefits of Each Proposal Response.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
16
Identify the Drawbacks, Cons, and Concerns of Each Proposal Response.
IMPORTANT: Evaluators should not assign scores during
independent review of proposal.
Steps to the Consensus Scoring Evaluation Session
The Evaluation team continues until all desired elements of discussion are
exhausted by all evaluators.
Each Member indicates points of information where they need clarification
and ask questions to fill such voids, when possible, with other evaluation team
members.
Each Member identifies the Pros and Benefits of each proposal.
Each Member identifies the Cons, Costs, and Concerns of each proposal.
Each Member communicates their assessment of the Overall Quality of each
Proposal Response based upon each criterion.
Only after all Proposals are discussed, then each proposal response is scored
by the team based upon all criteria.
The Facilitator records the scores for Documentation and Reporting.
Role of the Facilitator
The DOP CO serves as the Facilitator for the Evaluation Committee. The Facilitator
provides structure and ensures that all evaluators participate and contribute to the
process in a balanced and generous manner. The Facilitator will lead the consensus
scoring and evaluation session with the Evaluation Team and remind evaluators to
focus strictly on the evaluation criteria. The Facilitator will be responsible for ensuring
that the evaluation plan is followed and that scoring decisions are sound and
defensible.
The Facilitator will monitor and ensure that an objective stance is upheld throughout
the entire evaluation process. Facilitators will work closely with the evaluation team to
ensure that all needed clarification is obtained regarding the proposals under review,
and will resolve any compliance issues, and performs the final ranking of the proposals.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
17
Documentation Steps for Consensus RFP Evaluations
Individuals selected as evaluators are responsible for the execution of the
technical components evaluation as defined in this manual. Specifically,
evaluators will apply the pre-established procedures and criteria to
determine if each proponent’s technical components are responsive and
rate each of the evaluation categories. An RFP Evaluation Training is provided at the
onset of each project by the Department of Procurement for evaluators, and again
prior to the proposals being released to the evaluators.
Templates are provided for each of the evaluators to provide the following
documentation for consensus RFP evaluations.
Technical Evaluation matrix (one score for each company)
Cost Evaluation Scores (one score for each company)
Total Final Score Matrix
Evaluation Session Notes by Facilitator
IMPORTANT: The facilitator/procurement staff will take notes during
Collaborative Evaluation Session
R F P E V A L U A T I O N T R A I N I N G M A N U A L
18
Helpful Hints and Best Practices
1. Make every effort to differentiate the quality among
proposals reviewed.
2. RFP Evaluations for each contract are expected to be
complete within 10 days.
3. Keep in mind that you are making a decision that has an impact on
significant dollars, often millions- be dedicated and focused.
4. Cost proposals should be strictly evaluated based on a formula.
5. Technical proposals should be evaluated strictly by the already developed
criteria.
6. Benchmark costs to ensure reasonableness of proposed costs.
7. Thoroughly read the evaluation criteria.
8. Attend every scheduled evaluation team meeting and contribute to the
discussion. This is mandatory.
Next Steps: The Recommendation
Once the evaluation process is complete and the most responsive and responsible
proponent(s) has been identified, the following next steps can be anticipated to get to
contract award:
1. The Project Manager and respective AGM write a supporting letter for the
recommendation by the Evaluation Committee.
2. The General Manager sponsors the recommendation and sends to the
Chief Procurement Officer.
3. City Council and Mayor must approve the recommendation in order for
the City to enter into the contract with the proponent.
4. The Contract must be signed by the GM, Law, the CPO, the Mayor and
the Clerk.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
19
Frequently Asked Questions
The following are Frequently Asked Questions that may be useful as you are involved
in the RFP Evaluation process:
1. Can Evaluators bring their notes to the Consensus
Meeting?
Yes. Evaluators may bring their notes to facilitate discussion
during the Consensus Evaluation and Scoring Session.
2. If the evaluation team cannot reach a consensus, if averaging allowed?
No. Averaging is not done as a part of the collaborative scoring process.
Evaluators must agree to a score based on discussions regarding the pros
and cons of each proposal in order to reach a consensus score.
3. Are we allowed to take notes during our independent review of the
proposals?
Yes.
4. Can you discuss the RFP with subject matter experts not on the
evaluation panel?
No. Evaluators are prohibited from discussing the contents of RFP or any
procurement activities with anyone outside of the evaluation committee.
5. Can you take the proposals home to review?
Yes. However, evaluators must ensure that the proposals remain secure as
the content is confidential.
6. How many should comprise the evaluation team?
The evaluation team should have between three and five members.
7. When are reference checks conducted and by whom?
The APU Contracting Officer will conduct the reference check after the
proposals are received. The results will be distributed to the evaluation team
prior to the collaborative scoring session.
8. When are scores for OCC and Financial criteria provided?
The scores for OCC and financial criteria are provided by the DOP Contracting
Officer after the conclusion of the collaborative scoring.
R F P E V A L U A T I O N T R A I N I N G M A N U A L
20

Contenu connexe

Tendances

Proposal writing management
Proposal writing managementProposal writing management
Proposal writing managementCMCSoft Ltd. Co.
 
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATION
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATIONCONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATION
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATIONChester, Ryan Bourne MSc., MBA, PMP
 
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPs
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPsAssessing Probability, Risk and Cost in Responses to U.S. Federal RFPs
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPsVisibleThread
 
5. source selection_101
5. source selection_1015. source selection_101
5. source selection_101Colorado PTAC
 
7. past performance_seminar
7. past performance_seminar7. past performance_seminar
7. past performance_seminarColorado PTAC
 
RFP 2.0 - The "Do-It-Yourself" RFP Tool
RFP 2.0 - The "Do-It-Yourself" RFP ToolRFP 2.0 - The "Do-It-Yourself" RFP Tool
RFP 2.0 - The "Do-It-Yourself" RFP ToolCielo
 
Shushant CV_Presales_Bid Management
Shushant CV_Presales_Bid ManagementShushant CV_Presales_Bid Management
Shushant CV_Presales_Bid ManagementShushant Singh
 
SMEF2010 Request For Proposal Management Ask The Right Questions And Choos...
SMEF2010 Request For Proposal Management    Ask The Right Questions And Choos...SMEF2010 Request For Proposal Management    Ask The Right Questions And Choos...
SMEF2010 Request For Proposal Management Ask The Right Questions And Choos...Harold van Heeringen
 
SBA PResentation final
SBA PResentation finalSBA PResentation final
SBA PResentation finalSimone Sander
 
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...Government Contract Pricing Summit
 
Preparation of project report for bank finance
Preparation of project report for bank financePreparation of project report for bank finance
Preparation of project report for bank financeRevanth Rao
 
Bathu Dun Corporate Overview Feb 2012
Bathu Dun Corporate Overview Feb 2012Bathu Dun Corporate Overview Feb 2012
Bathu Dun Corporate Overview Feb 2012ravics
 
Six Sigma Leaders For Today Presentation Slides (Six Sigma)
Six Sigma Leaders For Today Presentation Slides (Six Sigma)Six Sigma Leaders For Today Presentation Slides (Six Sigma)
Six Sigma Leaders For Today Presentation Slides (Six Sigma)JoanMullins
 
APMP Foundation Certification Session 2 - Proposal Planning
APMP Foundation Certification Session 2  - Proposal PlanningAPMP Foundation Certification Session 2  - Proposal Planning
APMP Foundation Certification Session 2 - Proposal PlanningAbhijit Majumdar CP.APMP
 
Proposal Management: Best Industry Practices
Proposal Management: Best Industry PracticesProposal Management: Best Industry Practices
Proposal Management: Best Industry PracticesLohfeld Consulting Group
 
Preparation of Project Report
Preparation of Project ReportPreparation of Project Report
Preparation of Project Reportguest821cf8f
 
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...Top 10 Government Proposal Management Challenges and How to Overcome Them Web...
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...OST Global Solutions
 
Capture Management Overview
Capture Management OverviewCapture Management Overview
Capture Management OverviewPatricia Sego
 

Tendances (20)

Proposal writing management
Proposal writing managementProposal writing management
Proposal writing management
 
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATION
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATIONCONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATION
CONSULTING SERVICES PROCUREMENT AND SELECTION AUGUST 2013 PRESENTATION
 
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPs
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPsAssessing Probability, Risk and Cost in Responses to U.S. Federal RFPs
Assessing Probability, Risk and Cost in Responses to U.S. Federal RFPs
 
5. source selection_101
5. source selection_1015. source selection_101
5. source selection_101
 
7. past performance_seminar
7. past performance_seminar7. past performance_seminar
7. past performance_seminar
 
RFP 2.0 - The "Do-It-Yourself" RFP Tool
RFP 2.0 - The "Do-It-Yourself" RFP ToolRFP 2.0 - The "Do-It-Yourself" RFP Tool
RFP 2.0 - The "Do-It-Yourself" RFP Tool
 
Shushant CV_Presales_Bid Management
Shushant CV_Presales_Bid ManagementShushant CV_Presales_Bid Management
Shushant CV_Presales_Bid Management
 
SMEF2010 Request For Proposal Management Ask The Right Questions And Choos...
SMEF2010 Request For Proposal Management    Ask The Right Questions And Choos...SMEF2010 Request For Proposal Management    Ask The Right Questions And Choos...
SMEF2010 Request For Proposal Management Ask The Right Questions And Choos...
 
SBA PResentation final
SBA PResentation finalSBA PResentation final
SBA PResentation final
 
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...
Tips to Keep Pricing Professionals from Losing their Cool in the Summer Propo...
 
Preparation of project report for bank finance
Preparation of project report for bank financePreparation of project report for bank finance
Preparation of project report for bank finance
 
Bathu Dun Corporate Overview Feb 2012
Bathu Dun Corporate Overview Feb 2012Bathu Dun Corporate Overview Feb 2012
Bathu Dun Corporate Overview Feb 2012
 
Resume
ResumeResume
Resume
 
Six Sigma Leaders For Today Presentation Slides (Six Sigma)
Six Sigma Leaders For Today Presentation Slides (Six Sigma)Six Sigma Leaders For Today Presentation Slides (Six Sigma)
Six Sigma Leaders For Today Presentation Slides (Six Sigma)
 
Goods Procurement­
Goods Procurement­Goods Procurement­
Goods Procurement­
 
APMP Foundation Certification Session 2 - Proposal Planning
APMP Foundation Certification Session 2  - Proposal PlanningAPMP Foundation Certification Session 2  - Proposal Planning
APMP Foundation Certification Session 2 - Proposal Planning
 
Proposal Management: Best Industry Practices
Proposal Management: Best Industry PracticesProposal Management: Best Industry Practices
Proposal Management: Best Industry Practices
 
Preparation of Project Report
Preparation of Project ReportPreparation of Project Report
Preparation of Project Report
 
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...Top 10 Government Proposal Management Challenges and How to Overcome Them Web...
Top 10 Government Proposal Management Challenges and How to Overcome Them Web...
 
Capture Management Overview
Capture Management OverviewCapture Management Overview
Capture Management Overview
 

Similaire à Evaluation Training Guide

PART 1 Search RFP solicitations - IT Related, and write an analys.docx
PART 1 Search RFP solicitations - IT Related, and write an analys.docxPART 1 Search RFP solicitations - IT Related, and write an analys.docx
PART 1 Search RFP solicitations - IT Related, and write an analys.docxkarlhennesey
 
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docx
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docxHow to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docx
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docxpooleavelina
 
PROJ598—Contract and Procurement ManagementRFP Templat.docx
PROJ598—Contract and Procurement ManagementRFP Templat.docxPROJ598—Contract and Procurement ManagementRFP Templat.docx
PROJ598—Contract and Procurement ManagementRFP Templat.docxleahlegrand
 
An Inventory Control SystemYou sell seeds from a catalog, and bu.docx
An Inventory Control SystemYou sell seeds from a catalog, and bu.docxAn Inventory Control SystemYou sell seeds from a catalog, and bu.docx
An Inventory Control SystemYou sell seeds from a catalog, and bu.docxgalerussel59292
 
LMS RFP Template / Sample
LMS RFP Template / SampleLMS RFP Template / Sample
LMS RFP Template / SampleParadiso LMS
 
Best practices for a far 15 procurement part 1
Best practices for a far 15 procurement part 1Best practices for a far 15 procurement part 1
Best practices for a far 15 procurement part 1SacramentoNCMA Gold Rush
 
MGMT408—Contract and Procurement ManagementThe formats.docx
MGMT408—Contract and Procurement ManagementThe formats.docxMGMT408—Contract and Procurement ManagementThe formats.docx
MGMT408—Contract and Procurement ManagementThe formats.docxjessiehampson
 
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinal
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinalC:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinal
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinalMichele Roth
 
Hrd870grouppppresentationfinal
Hrd870grouppppresentationfinalHrd870grouppppresentationfinal
Hrd870grouppppresentationfinalMichele Roth
 
Real Stories from the RFP Process
Real Stories from the RFP ProcessReal Stories from the RFP Process
Real Stories from the RFP Processrshonerd
 
PROJ598—Contract and Procurement ManagementRFP Template.docx
PROJ598—Contract and Procurement ManagementRFP Template.docxPROJ598—Contract and Procurement ManagementRFP Template.docx
PROJ598—Contract and Procurement ManagementRFP Template.docxbriancrawford30935
 
MGMT408—Contract and Procurement ManagementRFP Template
MGMT408—Contract and Procurement ManagementRFP TemplateMGMT408—Contract and Procurement ManagementRFP Template
MGMT408—Contract and Procurement ManagementRFP TemplateMargenePurnell14
 
MGMT408—Contract and Procurement ManagementRFP Template.docx
MGMT408—Contract and Procurement ManagementRFP Template.docxMGMT408—Contract and Procurement ManagementRFP Template.docx
MGMT408—Contract and Procurement ManagementRFP Template.docxbuffydtesurina
 
Debriefs government point of view
Debriefs government point of viewDebriefs government point of view
Debriefs government point of viewjpeabody
 
PROJ598—Contract and Procurement ManagementYou have to make .docx
PROJ598—Contract and Procurement ManagementYou have to make .docxPROJ598—Contract and Procurement ManagementYou have to make .docx
PROJ598—Contract and Procurement ManagementYou have to make .docxwkyra78
 
RFP execution a practical guideline to improve selection process
RFP execution a practical guideline to improve selection processRFP execution a practical guideline to improve selection process
RFP execution a practical guideline to improve selection processMario Navarro
 
The Art of Planning and Writing Specs and Requirements--ISM 2010 Tanel
The Art of Planning and Writing Specs and Requirements--ISM 2010 TanelThe Art of Planning and Writing Specs and Requirements--ISM 2010 Tanel
The Art of Planning and Writing Specs and Requirements--ISM 2010 TanelThomas Tanel
 
Proposal and Bid Management Terms.pptx
Proposal and Bid Management  Terms.pptxProposal and Bid Management  Terms.pptx
Proposal and Bid Management Terms.pptxNavas Kilikkottu
 
RFQ-RFP Best Practices Workshop Jan2012.ppt
RFQ-RFP Best Practices Workshop Jan2012.pptRFQ-RFP Best Practices Workshop Jan2012.ppt
RFQ-RFP Best Practices Workshop Jan2012.pptAravindReddyKetham1
 

Similaire à Evaluation Training Guide (20)

RFP Evaluation Training
RFP Evaluation TrainingRFP Evaluation Training
RFP Evaluation Training
 
PART 1 Search RFP solicitations - IT Related, and write an analys.docx
PART 1 Search RFP solicitations - IT Related, and write an analys.docxPART 1 Search RFP solicitations - IT Related, and write an analys.docx
PART 1 Search RFP solicitations - IT Related, and write an analys.docx
 
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docx
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docxHow to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docx
How to Perfectly Construct an RFP in 8 StepsThe RFP (request for.docx
 
PROJ598—Contract and Procurement ManagementRFP Templat.docx
PROJ598—Contract and Procurement ManagementRFP Templat.docxPROJ598—Contract and Procurement ManagementRFP Templat.docx
PROJ598—Contract and Procurement ManagementRFP Templat.docx
 
An Inventory Control SystemYou sell seeds from a catalog, and bu.docx
An Inventory Control SystemYou sell seeds from a catalog, and bu.docxAn Inventory Control SystemYou sell seeds from a catalog, and bu.docx
An Inventory Control SystemYou sell seeds from a catalog, and bu.docx
 
LMS RFP Template / Sample
LMS RFP Template / SampleLMS RFP Template / Sample
LMS RFP Template / Sample
 
Best practices for a far 15 procurement part 1
Best practices for a far 15 procurement part 1Best practices for a far 15 procurement part 1
Best practices for a far 15 procurement part 1
 
MGMT408—Contract and Procurement ManagementThe formats.docx
MGMT408—Contract and Procurement ManagementThe formats.docxMGMT408—Contract and Procurement ManagementThe formats.docx
MGMT408—Contract and Procurement ManagementThe formats.docx
 
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinal
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinalC:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinal
C:\Users\Michele Roth\Documents\Hrd 870 Bronack\Hrd870grouppppresentationfinal
 
Hrd870grouppppresentationfinal
Hrd870grouppppresentationfinalHrd870grouppppresentationfinal
Hrd870grouppppresentationfinal
 
Real Stories from the RFP Process
Real Stories from the RFP ProcessReal Stories from the RFP Process
Real Stories from the RFP Process
 
PROJ598—Contract and Procurement ManagementRFP Template.docx
PROJ598—Contract and Procurement ManagementRFP Template.docxPROJ598—Contract and Procurement ManagementRFP Template.docx
PROJ598—Contract and Procurement ManagementRFP Template.docx
 
MGMT408—Contract and Procurement ManagementRFP Template
MGMT408—Contract and Procurement ManagementRFP TemplateMGMT408—Contract and Procurement ManagementRFP Template
MGMT408—Contract and Procurement ManagementRFP Template
 
MGMT408—Contract and Procurement ManagementRFP Template.docx
MGMT408—Contract and Procurement ManagementRFP Template.docxMGMT408—Contract and Procurement ManagementRFP Template.docx
MGMT408—Contract and Procurement ManagementRFP Template.docx
 
Debriefs government point of view
Debriefs government point of viewDebriefs government point of view
Debriefs government point of view
 
PROJ598—Contract and Procurement ManagementYou have to make .docx
PROJ598—Contract and Procurement ManagementYou have to make .docxPROJ598—Contract and Procurement ManagementYou have to make .docx
PROJ598—Contract and Procurement ManagementYou have to make .docx
 
RFP execution a practical guideline to improve selection process
RFP execution a practical guideline to improve selection processRFP execution a practical guideline to improve selection process
RFP execution a practical guideline to improve selection process
 
The Art of Planning and Writing Specs and Requirements--ISM 2010 Tanel
The Art of Planning and Writing Specs and Requirements--ISM 2010 TanelThe Art of Planning and Writing Specs and Requirements--ISM 2010 Tanel
The Art of Planning and Writing Specs and Requirements--ISM 2010 Tanel
 
Proposal and Bid Management Terms.pptx
Proposal and Bid Management  Terms.pptxProposal and Bid Management  Terms.pptx
Proposal and Bid Management Terms.pptx
 
RFQ-RFP Best Practices Workshop Jan2012.ppt
RFQ-RFP Best Practices Workshop Jan2012.pptRFQ-RFP Best Practices Workshop Jan2012.ppt
RFQ-RFP Best Practices Workshop Jan2012.ppt
 

Evaluation Training Guide

  • 1. HARTSFIELD-JACKSON ATLANTA INTERNATIONAL AIRPORT - CITY OF ATLANTA Aviation Procurement Unit RFPEvaluation TrainingManual
  • 2. A V I A T I O N P R O C U R E M E N T U N I T RFP Evaluation Training Manual © 2009 City of Atlanta. Aviation Procurement Unit 6000 North Terminal Parkway • Atrium Suite 4000 Atlanta, GA 30320 Phone 404.530.5408 • Fax 404.905.1589
  • 3. Table of Content Introduction................................................................................ 1 Definition ................................................................................... 1 The RFP as a Solution .............................................................. 2 When to Use an RFP Approach?.................................................... 2 What is the Best Way to Handle Criteria for RFP Evaluations? ...... 3 How Do We Develop an Evaluation Strategy?................................ 3 What are Examples of Standard Criteria in a Technical Proposal?. 4 Overview of the Evaluation Process.......................................... 6 Standards of Conduct While Serving as an Evaluator..................... 6 What is Expected from Evaluation Committee Members? .............. 7 Oral Interviews ................................................................................ 7 Reference Checks .......................................................................... 8 Ensure Security and Confidentiality ................................................ 9 Approach to Evaluating Proposals .......................................... 10 Evaluation Scoring Guidelines ...................................................... 10 When is a Proposal Non-Responsive?.......................................... 11 Approach to Evaluating Technical Proposals................................ 11 Approach to Evaluating Cost Proposals........................................ 13 Consensus Evaluation and Scoring ........................................ 15 Benefits of Consensus Evaluation and Scoring ............................ 15 Preparing for the Consensus Evaluation Session......................... 15 Steps to the Consensus Scoring Evaluation Session ................... 16 Role of the Facilitator .................................................................... 16 Documentation Steps for Consensus RFP Evaluations ................ 17 Helpful Hints and Best Practices............................................. 18 Next Steps: The Recommendation ......................................... 18 Frequently Asked Questions ................................................... 19
  • 4. T H E R F P A S A S O L U T I O N 1 Introduction A Best Value Award is one which optimizes quality, performance, cost, time, and efficiency in a fair manner to the contractor community within the contract. ids and quotes cannot always address the needs of the Department of Aviation (DOA). Generic specifications may not be available or difficult or impossible to draft and conventional evaluation for award based on lowest cost bid may not get the product or service required. Many high tech products and complex services cannot be obtained by conventional bidding. The Request for Proposal (RFP) is a solicitation used for situations like these. Definition The RFP is a formal competitive sealed bid process. The RFP outlines the requirements of the Business Unit by describing the purpose, scope, description, minimum requirements or expectations, qualifications or capability of the proposers, evaluation criteria, and other requirements. In the RFP response, the vendor offers a solution for the particular need described in the RFP. The RFP is evaluated according to predetermined weighted standards. After completion of the RFP process negotiations may, in some circumstances, be utilized to secure more advantageous terms or reduced cost. Volume 1 B
  • 5. R F P E V A L U A T I O N T R A I N I N G M A N U A L The RFP as a Solution When to Use an RFP Approach? A successful Request for Proposals (RFP) requires much planning. No two RFP’s are alike and preparing one can be a difficult task. However, a well-written RFP can alleviate many problems. Business Units are encouraged to involve the Aviation Procurement Unit (APU) early in the development of the RFP. APU can provide assistance and valuable advice. The following are typical instances when an RFP is the best approach; When the use of competitive sealed bidding is either not practible or not advantageous to the DOA. When we are not sure of the best solution and multiple options exist. When performance-based incentive contracts are used. When complex factors in addition to price are very important. When there are several complex requirements. When it is a new program/ project that has not been tested. When it is desirable to find out if there are alternative options to what is normally performed. A RFP is one of several procurement tools. APU will work with the Business Unit Project Manager to determine whether it is the best tool for the goods or services that are needed. The Chief Procurement Officer (CPO), upon receiving recommendation from APU and the Contracting Officer (CO), will ultimately determine if the RFP process is appropriate. NOTE: The bid process is the default solicitation option.
  • 6. T H E R F P A S A S O L U T I O N 3 What is the Best Way to Handle Criteria for RFP Evaluations? It is important to identify all evaluation criteria and their relative importance, including price, early in the RFP development stages. These criteria will be the only way to properly evaluate the proposals and assure that the awarded proposals meets all the requirements of the solicitation. Criteria not specified in the RFP cannot be used for evaluating the proposals. Begin by making a detailed list of the most important aspects of the services or goods required, including cost. Each item on your list is a potential evaluation criterion. Arrange the list in the sequence of what is most important. Next, assign a point factor to each criterion based on its relative importance. The most important items should be evaluated heavier and have more points available. Points assigned to each criterion are included in the RFP. Including points makes offerors aware of which items are relatively more important than others can influence an offeror in the preparation of their RFP response. All Evaluation Criteria MUST be established prior to the issuance of the RFP. Criteria should be clearly linked to desirable outcomes. Be Clear and Concise in language to avoid confusion. Evaluation Information MUST be documented, dated, and secured throughout the evaluation process. How Do We Develop an Evaluation Strategy? The following should be considered in developing your evaluation strategy. 1. Identify (eliminating) Minimum Qualifications These are requirements that a vendor must meet in order to accomplish the work outlined in the RFP. They may include such things as proper licensing or accreditation and special insurance or bonding. They are evaluated on a strictly pass-or-fail basis. Make a list of the things that will be absolutely required for a successful offeror to have in order to enter into a contract. Do not include “desired” things or items that could be obtained by an offeror at a later time prior to contract award. Generally, if an offeror fails on any portion of the mandatory requirements, their proposal will be rejected, so put careful thought to this step. 2. Define Criteria Develop Statement of Intent or Purpose and Scope of Work - The Statement of Intent or Purpose is a description of the general type of
  • 7. T H E R F P A S A S O L U T I O N 4 service or goods required. The Scope of Work is a general summary of the work to be performed by a contractor. Developing these brief descriptions first will assist you to begin organizing your thoughts and help you decide on proper evaluation factors. Decide importance of Oral Interviews/ Presentations - To properly evaluate proposals, oral presentations may be scheduled to answer questions for evaluation committee members. After consultation with the CO, only those firms that are potentially acceptable are invited to participate in oral presentations. The offeror’s original proposal cannot be changed in any aspect at the oral presentation. The oral presentation is only to allow offerors to clarify portions of their proposal. Incorporate Reference Checks into Industry Experience - The purpose of contacting references is to verify the corporate capabilities and prior performance of the proponent and the qualifications of proposed project personnel. Reference checks are made by telephone. The APU Contracting Officer designated to assist the Evaluation Team will contact the references identified by the proponent. The results of the reference checks are compiled and provided to each evaluator to assist in scoring specific criteria. All reference questions must be relevant to the scope. The same questions must be asked to all references for all proponents and should be structured to assist the evaluators as they review information from each reference and to support follow-up questioning on selected reference responses. Decide importance of Site Visits –The purpose of the site visit is to acquaint the vendors with the conditions under which the work must be performed. 3. How important is Cost?: Cost to Technical Score Ratio; 4. Determine distribution of points among each criterion. What are Examples of Standard Criteria in a Technical Proposal? The following are examples of standard criteria in a Technical Proposal. I. Corporate Experience: What are the proponents’ past performance and experience in similar types of contracts? What is the experience level of the essential subcontractors?
  • 8. T H E R F P A S A S O L U T I O N 5 II. Project Organization and Staffing: What is the staffing approach? What are the qualifications and past performance of the personnel? III. Implementation Approach: What is the adequacy of their work breakdown structure/ project management? Is it Comprehensive? Reasonable? Proposed Deliverables? Preparation Plan? IV. Operation/ Management Approach: Plan for Regular/ Daily Operations? Approach to Quality Control and Assurance? What are the Commitments to Performance Standards? What are the Plans for Maintenance? What are the Contingency Procedures in the event of an emergency? V. Corporate and General Capacity: What is the overall capacity of the organization? Do they have a plan to sufficiently staff the contract project? How long has the company been operating? What is the turnover ratio?
  • 9. R F P E V A L U A T I O N T R A I N I N G M A N U A L Overview of the Evaluation Process Evaluation of the RFP responses is best accomplished in a team effort. The evaluation committee is comprised of the evaluation team, the APU Contracting Officer and the Department of Procurement (DOP) Contracting Officer as the Facilitator. NOTE: APU facilitates and coordinates processes between the Department of Procurement and evaluators. Rules and regulations that are applicable to the evaluation process must be followed throughout the entire process. The following is a brief overview of the evaluation process: Evaluation committee members independently review proposals based upon the specific evaluation criteria. Technical and cost scores are assigned applying the designated evaluation criteria. Oral interviews are conducted, if necessary. Negotiations are conducted with the winning proponent. Standards of Conduct While Serving as an Evaluator The evaluation team should be made up of individuals with varied talents and expertise to assure impartiality. Team members need to be aware of the possibility of an extended time commitment before agreeing to be a participant. The following standards of conduct must be adhered to while serving as an evaluator: No Conflicts of Interest No solicitation or acceptance of gifts or anything of value Integrity and Honesty during scoring and communications Proper handling and discretion of all information; No Disclosure during process Practice Ethical Behavior
  • 10. R F P E V A L U A T I O N T R A I N I N G M A N U A L 7 What is Expected from Evaluation Committee Members? The evaluation committee members play a significant role in the decision to select a contractor, which results in the award of a contract. Accordingly, it is imperative that everyone understands what makes a good evaluation committee member. The following are standards established by DOP and APU; 1. The member has knowledge or experience regarding the RFP subject matter. 2. The member has sufficient time to dedicate in order to effectively evaluate the proposals. Such time is dedicated to reading all proposal related materials extensively. Mandatory attendance to all evaluation-related meetings: trainings, evaluation sessions, oral interviews, debriefings, criteria planning. 3. Conduct an independent review. Identify potential deficiencies in a proposal that could result in a determination of non-responsiveness or non-responsibility. Limit communication regarding the RFP project to those DOA and DOP staff who are directly assigned to the project. 4. Refrain from any communications with potential vendors/ contractors. 5. Determine whether or not oral interviews will be necessary. Oral interviews should follow group discussions Oral interviews should ALWAYS precede scoring. Oral Interviews In complicated services, it may be beneficial to require offerors to make an oral presentation. Discussions may be conducted with responsible and responsive offeror’s who submit proposals determined to be reasonably susceptible of being selected for award. The purposes of Oral Interviews are as follows: Allows proponents to clarify on areas that may be unclear in their proposals. Possibly reduces the likelihood of misunderstandings.
  • 11. R F P E V A L U A T I O N T R A I N I N G M A N U A L 8 The evaluation committee members determine whether or not oral interviews will be necessary. Oral interviews should follow group discussion, and should ALWAYS precede scoring. Reference Checks Reference Checks will be conducted with prior customers of the proponent. Reference Checks may also include references of key personnel. A standard form is used to collect responses. However, additional subject matter questions may be included when requested. When additional reference questions are to be included as part of the evaluation, the following information is required: List of questions for references. Instructions to be given to the reference including a numerical scale to be used in rating the offeror. References should be contacted only once. The APU Contracting Officer will conduct the reference check, which may include a conference call with all members of the evaluation team present when there are subject matter specific questions. A Reference Analysis will be provided to the Evaluation Team prior to the collaborative scoring and evaluation session. The Reference Verification Form includes factors/ratings for technical performance, management performance, and customer satisfaction. The references will be asked to rate their level of satisfaction from 1-5. Examples of Questions include: Were the major tasks/milestones/ deliverables on schedule? Were they able to identify and solve problems expeditiously? What was the effectiveness and reliability of key personnel? Were they able to recruit and maintain qualified personnel? Were they able to effectively manage essential subcontractors? What was the overall performance in planning, scheduling and monitoring? How would you rate their overall technical performance? How would you rate their overall management performance?
  • 12. R F P E V A L U A T I O N T R A I N I N G M A N U A L 9 How would you rate their ability to be cooperative, business-like and concerned with the interests of the customer? Would you do business with this vendor/ contractor again? How would you rate your level of satisfaction from 1-5? Ensure Security and Confidentiality Throughout all phases of the evaluation, the confidentiality and security of proposals and the scoring process must be maintained. To ensure confidentiality and security, evaluation sessions will be closed to the public and staff who are not supporting the Evaluation Team. The evaluators shall not discuss the contents of proposals or the procurement activities with any persons outside of the evaluation team or Procurement staff. All evaluators and all other staff involved in the evaluation effort must strictly adhere to the following requirements. Evaluators must keep proposals in a secure place. Evaluators are not permitted to discuss the procurement or evaluation process with any persons outside of the evaluation team or procurement staff. Evaluators shall not communicate the scoring outcomes or content of proposals and shall not disclose the status of any proposal. Cost components shall be evaluated upon the completion of the scoring of the technical components.
  • 13. R F P E V A L U A T I O N T R A I N I N G M A N U A L 10 Approach to Evaluating Proposals All proposals shall be evaluated in accordance with the City of Atlanta’s Procurement Code and the criteria specified in the RFP on the Percentage Evaluation Form and considering the information required to be submitted in each Proposal. The evaluators will independently read and assess compliance with the criteria identified in the RFP to assess the completeness, quality, and desirability of proponent responses. All members of the Evaluation Team will document their findings for evaluation criteria within each of these areas. Each criterion is independently reviewed by evaluators, documenting the pros and cons of each. Review reflects individual, independent evaluations of a proposal and response to criteria. Criteria include questions or items for consideration for evaluators to use as a guide in discussion during the collaborative scoring and evaluation session. The Evaluation Team should not score the criteria during their independent evaluation. Evaluation Scoring Guidelines Each technical proposal will be evaluated against a set of pre-determined criteria to assess the degree to which it meets that criterion. Compliance with requirements will be assessed as a point score on a scale from 1 to 10. Technical criteria will be scored as shown below: 8-10: Outstanding: Proponent’s proposal exceeds expectation and demonstrates an excellent ability to reach the goals and objectives of the procurement. 5-7: Highly Effective: Proponent’s proposal demonstrates a good ability to reach the goals and objectives of the procurement. 3-4: Effective: Proponent’s proposal demonstrates a fair ability to reach the goals and objectives of the procurement. 1-2: Marginally Effective: Proponent’s proposal demonstrates a minimal ability to reach the goals and objectives of the procurement. Scores will be adjusted to reflect the weight assigned to each criterion. Each criterion must be reviewed according to the requested items of the RFP and scored based on the scoring definitions above. Only the Criteria in the Percentage Evaluation Form will receive scores. All notes taken regarding any proposal are subject to the Open Records Act.
  • 14. R F P E V A L U A T I O N T R A I N I N G M A N U A L 11 Note Any Proponents’ Proposal that does not meet the minimum requirements of the RFP should be identified as this may deem them non-responsive. When is a Proposal Non-Responsive? Only the Department of Procurement has the authority to determine whether a proposal is non-responsive or non-responsible. A Proposal will be determined to be non-responsive when the proposer is not responsive to the mandatory requirements clearly established within the RFP. If an evaluator at any time has a concern with the proponents’ proposal meeting the stated requirements, they should immediately notify the Facilitator and Procurement staff. Approach to Evaluating Technical Proposals The purpose of this phase is to measure the individual merits of the technical components of the proposal against pre-established criteria. Members of the Evaluation Team will review all Technical Proposals that pass the Mandatory Technical Requirements. Evaluators are instructed to read through the proposal Executive Summary (if present) before beginning to evaluate and score detailed criteria. The material in the proposal introduction should provide all evaluators with a broad understanding of the entire proposal. In assessing individual responses within a section, evaluators may elect to review related topics within other sections of the proposal. Evaluators are restricted to evaluating information contained within the proposal. Information not part of the proposal may not be considered. Evaluators will proceed according to the following steps to evaluate responses to each criterion: 1. Review the appropriate section of the RFP. 2. Locate the section(s) of the proposal where the criterion is addressed. 3. For each criterion, note the RFP sections referenced in this document. 4. Review and evaluate section(s) of the proposal. Evaluators should promptly notify the CO if there is a problem with the Proposal.
  • 15. R F P E V A L U A T I O N T R A I N I N G M A N U A L 12 5. For each criterion, evaluate and assess the criterion based on the proponent’s overall response to the requirements indicated for the criterion (see step 8). Some of the criteria have additional questions that may be considered in the evaluation. 6. Evaluate how well the proponent’s responses in the referenced sections correlate with other pertinent sections of the RFP and the overall approach taken to address the technical components of the proposal. 7. Evaluate the criterion based on all information available that pertains to it directly or indirectly, including reference checks. 8. Develop a list of pros and cons for the criterion based on the evaluation of the proponent’s capability to meet that criterion. Evaluators are encouraged to request technical support from the CO or from the Department of Procurement in preparation for criteria documentation. Training will be provided at the inception of each Evaluation Committee, and prior to proposal distribution. WhatdoesTechnicalScoringLookLike? Weight Criteria Company 1 Company 2 Company 3 20% Organization/Resumes of Key Personnel 10 3 1 30% Overall Experience, Qualifications and Performance of Previous Similar Projects 9 7 4 20% Strategic Approach To Increasing Federal Assistance to the City through Government Agencies and Congress 8 6 3
  • 16. R F P E V A L U A T I O N T R A I N I N G M A N U A L 13 Approach to Evaluating Cost Proposals After technical aspects of proposal responses are initially reviewed, then cost proposals are reviewed. Cost proposals may reflect price escalation, deceleration due to performance, time, CPI, etc. may be applied. Cost Evaluation must be independent of the Technical Evaluation. A Clear Formula must be used to Calculate Cost Scores. Scores should be directly linked solely to costs proposed on a formulaic basis. The cost proposal/ financial offer must be taken at face value when the cost proposal is evaluated. Note Scores are documented only in the Evaluation and Collaborative Scoring Session by the Facilitator/Procurement Staff. Traditionally, the Lowest Fee Offer receives the maximum points possible (10). In the event that we are soliciting a revenue generating proposal, the highest offer would receive the maximum points. Points are distributed based upon percentile in which the fee offer is found in relationship to the lowest offeror. Example Cost Offer Points Cost Proposal 1 $780k 10 Lowest Offer Cost Proposal 2 $1.3 mil 6 780k/1.3mil = .6 Cost Proposal 3 $1.8 mil 4.3 780k/1.8 mil = .43 Formula Used: Lowest Cost Proposal Offer/ Cost Proposal Offer at Hand = Percentage Relationship to Lowest Offer
  • 17. R F P E V A L U A T I O N T R A I N I N G M A N U A L 14 WhatAreSomeExamplesof CostProposals? The following are examples of what the cost proposals may read like for the types of RFP’s and resulting contracts: Type of Contract Suggested wording… Scenario Example Performance Based Please propose Base Costs and Incentives associated with performance beyond mandatory requirements… Beyond 90% Customer Satisfaction Scores consecutively for one year will lead to a 3% bonus (e.g.., janitorial) Cost Savings Please propose Base Costs and Incentives associated with cost savings beyond mandatory requirements… Company X will receive 10% of all cost savings beyond performance guarantee (e.g.., workers’ compensation)
  • 18. R F P E V A L U A T I O N T R A I N I N G M A N U A L 15 Consensus Evaluation and Scoring Consensus Scoring consists of the Entire Evaluation Committee participating in Joint Discussion, Assessment, and Scoring of Technical and Cost Criteria in order to develop one unified score for each criterion. The evaluators are allotted five (5) days to independently review the proponents’ proposals. Next, the evaluation committee will meet to evaluate the proposals, discussing the pros and cons of each proposal as well as clarifying their assessment. Benefits of Consensus Evaluation and Scoring While the benefits of consensus evaluation and scoring are vast, the following portrays the immediate results; Team Discussion leads to greater Sharing of Each Other’s Expertise in Cross- Functional Projects, which are quite common in DOA RFPs Generates Clarification where Misunderstandings May Exist Ensures that All evaluators are truly Engaged in the Evaluation process Encourages deeper Probing, Questioning, Discussion, and Analysis of the content of Proposals in response to the Evaluation Criteria A Unified Understanding of Responses to All Proposals is reached Eliminates discrepancies among scores Overall, the result is a more Comprehensive and Accurate Assessment of the proposals Preparing for the Consensus Evaluation Session In preparing for the consensus evaluation session, evaluation team members should review thoroughly and comprehensively all proposal responses, repeating review of responses where necessary. Keep in Mind How the Information Relates Specifically to the stated Criteria in the RFP. Identify All Points of Information in the Responses that Need Clarification Identify the Pros and Benefits of Each Proposal Response.
  • 19. R F P E V A L U A T I O N T R A I N I N G M A N U A L 16 Identify the Drawbacks, Cons, and Concerns of Each Proposal Response. IMPORTANT: Evaluators should not assign scores during independent review of proposal. Steps to the Consensus Scoring Evaluation Session The Evaluation team continues until all desired elements of discussion are exhausted by all evaluators. Each Member indicates points of information where they need clarification and ask questions to fill such voids, when possible, with other evaluation team members. Each Member identifies the Pros and Benefits of each proposal. Each Member identifies the Cons, Costs, and Concerns of each proposal. Each Member communicates their assessment of the Overall Quality of each Proposal Response based upon each criterion. Only after all Proposals are discussed, then each proposal response is scored by the team based upon all criteria. The Facilitator records the scores for Documentation and Reporting. Role of the Facilitator The DOP CO serves as the Facilitator for the Evaluation Committee. The Facilitator provides structure and ensures that all evaluators participate and contribute to the process in a balanced and generous manner. The Facilitator will lead the consensus scoring and evaluation session with the Evaluation Team and remind evaluators to focus strictly on the evaluation criteria. The Facilitator will be responsible for ensuring that the evaluation plan is followed and that scoring decisions are sound and defensible. The Facilitator will monitor and ensure that an objective stance is upheld throughout the entire evaluation process. Facilitators will work closely with the evaluation team to ensure that all needed clarification is obtained regarding the proposals under review, and will resolve any compliance issues, and performs the final ranking of the proposals.
  • 20. R F P E V A L U A T I O N T R A I N I N G M A N U A L 17 Documentation Steps for Consensus RFP Evaluations Individuals selected as evaluators are responsible for the execution of the technical components evaluation as defined in this manual. Specifically, evaluators will apply the pre-established procedures and criteria to determine if each proponent’s technical components are responsive and rate each of the evaluation categories. An RFP Evaluation Training is provided at the onset of each project by the Department of Procurement for evaluators, and again prior to the proposals being released to the evaluators. Templates are provided for each of the evaluators to provide the following documentation for consensus RFP evaluations. Technical Evaluation matrix (one score for each company) Cost Evaluation Scores (one score for each company) Total Final Score Matrix Evaluation Session Notes by Facilitator IMPORTANT: The facilitator/procurement staff will take notes during Collaborative Evaluation Session
  • 21. R F P E V A L U A T I O N T R A I N I N G M A N U A L 18 Helpful Hints and Best Practices 1. Make every effort to differentiate the quality among proposals reviewed. 2. RFP Evaluations for each contract are expected to be complete within 10 days. 3. Keep in mind that you are making a decision that has an impact on significant dollars, often millions- be dedicated and focused. 4. Cost proposals should be strictly evaluated based on a formula. 5. Technical proposals should be evaluated strictly by the already developed criteria. 6. Benchmark costs to ensure reasonableness of proposed costs. 7. Thoroughly read the evaluation criteria. 8. Attend every scheduled evaluation team meeting and contribute to the discussion. This is mandatory. Next Steps: The Recommendation Once the evaluation process is complete and the most responsive and responsible proponent(s) has been identified, the following next steps can be anticipated to get to contract award: 1. The Project Manager and respective AGM write a supporting letter for the recommendation by the Evaluation Committee. 2. The General Manager sponsors the recommendation and sends to the Chief Procurement Officer. 3. City Council and Mayor must approve the recommendation in order for the City to enter into the contract with the proponent. 4. The Contract must be signed by the GM, Law, the CPO, the Mayor and the Clerk.
  • 22. R F P E V A L U A T I O N T R A I N I N G M A N U A L 19 Frequently Asked Questions The following are Frequently Asked Questions that may be useful as you are involved in the RFP Evaluation process: 1. Can Evaluators bring their notes to the Consensus Meeting? Yes. Evaluators may bring their notes to facilitate discussion during the Consensus Evaluation and Scoring Session. 2. If the evaluation team cannot reach a consensus, if averaging allowed? No. Averaging is not done as a part of the collaborative scoring process. Evaluators must agree to a score based on discussions regarding the pros and cons of each proposal in order to reach a consensus score. 3. Are we allowed to take notes during our independent review of the proposals? Yes. 4. Can you discuss the RFP with subject matter experts not on the evaluation panel? No. Evaluators are prohibited from discussing the contents of RFP or any procurement activities with anyone outside of the evaluation committee. 5. Can you take the proposals home to review? Yes. However, evaluators must ensure that the proposals remain secure as the content is confidential. 6. How many should comprise the evaluation team? The evaluation team should have between three and five members. 7. When are reference checks conducted and by whom? The APU Contracting Officer will conduct the reference check after the proposals are received. The results will be distributed to the evaluation team prior to the collaborative scoring session. 8. When are scores for OCC and Financial criteria provided? The scores for OCC and financial criteria are provided by the DOP Contracting Officer after the conclusion of the collaborative scoring.
  • 23. R F P E V A L U A T I O N T R A I N I N G M A N U A L 20