7. Don't avoid significant issues which apply to the project or potential problems which may be relevant to the project. It is better to take a stand and discuss a process for dealing with anticipated problems than to avoid these questions.

8. Avoid inflated rhetoric or impossible promises.

9. Avoid unsupported subjective arguments.

10. Do not assume that the reader will be intimately familiar with the subject.

11. Sequence the components of the proposal in a logical manner.

12. Carefully review, edit, and proofread -- again and again . Get others to help, as in another opinion in a medical manner. Avoid errors in grammar, spelling, math, and maintain a clean overall appearance.

13. A proposal should be readable, should not be missing pages, and should be written in the same consistent style throughout.

SOME WRITING TIPS

by David R. Krathwohl

Obvious errors in writing undermine other evidence of competence. If basic English is a weakness, that is beyond our scope. Ask a friend for help or hire an editor. There are, however, simple things each of us can do to improve our writing.

Assume the writer's task is that of capturing the attention of busy but committed people, holding it, and leading them to the important points in the proposal. Make their reading easier by organizing ideas clearly. Increase the impact of what they are reading by suitable direct and simple language.

Organizing

1) Make the structure of the proposal clear.Use a variety of "road signs" to guide the reading and to highlight important points. Foreshadow what is coming and indicate what has been. Techniques for doing this include headings, marginal notes, sectional introductions and prefaces, summaries and appendices, outlines, charts and diagrams. Overusing them however, clutters the visual field. Don't!

Look for large, black sections of text. Break them up with paragraphs and headings. If titles and subtitles are difficult to assign or do not sequence properly, refine the organization of the text.

Diagrams and arrows can help show the flow of ideas and highlight important points. Be careful of boxes. If possible, leave the sides open so the reader has visual entry. Some people read around closed boxes, intending to come back, but then forget.

2) Make the proposal easy to skim.Clear organization with distinct "road signs" eases skimming. In addition:

Set a topic sentence into every paragraph. If an important topic sentence does not begin the paragraph, show where it is embedded with underlining, italics or boldface type.

Use white space to set off and highlight significant items.

Set parallel structures and comparisons side by side, if possible, to make the correspondences self-evident.

Use white spaces to provide visual relief and to frame the text pleasingly. Do not go overboard, however. Keep within the page allowance, if there is one.

Skim the proposal yourself, or better yet, ask someone for whom the proposal is new to skim it. Assure that the "road signs" lead the reader to the correct meaning.

3) Make transitions smoothly.Do not let the reader get lost at junction points. Proper sequencing, clear reference to earlier discussion, and constructive reasoning from such references join up with selective repetition of key phrases and words to assure easy shifts of perspective between sections, paragraphs and even sentences.

Direct, Concise Language

4) Use active verbs and simple constructions.Active verbs bring lucidity to sentences. Complex, passive constructions diminish the intensity of the communication by leading the reader into grammatical bottlenecks, thus breaking his or her concentration. By striking out words and phrases and rearranging the remainder (changing only a few words), good technical editors markedly clarify meaning. Try it!

Change passives to actives. For example, "It has been reported by the NIH that the India proposal was found to be complex," becomes, in the active voice: "The NIH found the India proposal complex."

Find extra "to be" verbs and eliminate them. For example, "The argument of social workers was that welfare is necessary," can be reduced to: "Social workers argued welfare is necessary." Another example, "Reductions in excess verbiage were more noticeable than reductions in clarity," can become: "Verbiage declined more noticeably than clarity."

5) Simplify sentences.When a page seems one big, black, unbroken wall of words, skim it for periods. Find sentences that go on and on and break them up. But keep an interesting rhythm of long and short sentences; don't let the writing become too choppy, too staccato.

6) Use concrete, "picture" language.Except when familiarity with technical terms must be shown, substitute everyday words for the more abstruse (eg. "end" instead of "terminate"; "begin", not "institute").

7) Convey liveliness and enthusiasm.This marks the proposer's commitment to the project.

These tips won't change a writer's style; by the time one is mature enough to be writing proposals, writing style is largely set. But posting a list like this where it will spark renewed effort can result in improvement. (Yes, it can! You should have seen this text's first drafts!)

Better still, make a list of personal weaknesses. Kathleen Dugdale's A Manual on Writing Research, 2nd Edition (Indiana University Bookstore, Bloomington, Ind., 1967) lists good suggestions for clear and effective writing plus eleven pages of troublesome words and constructions. Carolyn Mullins' Guide to Writing and Publishing in the Social and Behavioral Sciences (John A. Wiley and Sons, New York, N.Y., 1977) includes a useful chapter entitled "Revision of Text: Home Remedies for Prosaic Problems" with similar hints and lists. Simple and Direct: A Rhetoric for Writers by Jacques Barzun (Harper and Row, Inc., New York, N.Y., 1975) and similar books provide extensive suggestions. Find those that "strike home". Tape them to a corner of the desk or post them prominently on the wall where you can't avoid them.

David R. Krathwohl. How to Prepare a Research Proposal, 2nd Edition . 1977.

Consider the act of editing and revising a piece of writing as an act of quality control. If you (or your university representative) sign your name to it, you are asserting that the words and ideas are accurately expressed. Most technical people are not trained in editing and revising. Thus, they have no method or theory for undertaking revision of their proposals. Furthermore, you may do a better job of reading faculty proposals if you have a set of guidelines and a specific methodology to follow.

Here are some guidelines for editing and revising proposals (or any piece of professional communication):

1. Editing is the process of recognizing and identifying problems in the writing. Revising is the process of changing the items to conform with rules or to produce clearer, more effective writing.

2. Editing/revising can be done for mechanics, style, and structure. It is necessary to check mechanics. It is desirable (and sometimes necessary) to check style and structure.

3. Each type of editing/revising should be done independently.

4. It is not necessary to know the accurate grammatical term or concept to do effective editing and revising, but it is important to have adequate reference books for consultation. At minimum, there should be an up-to-date desk dictionary and a grammar handbook. (One possible source is Handbook of Technical Writing , Charles T. Brusaw, Gerald J. Alred, and Walter E. Oliu, St. Martin's Press, 1976.)

Editing/Revising for Mechanics :

This type of editing/revising is done word by word, with a concern for identifying individual items which do not conform with accepted grammatical or spelling rules. The goal of this process is uniformity and accuracy.In editing/revising for mechanics, look for the following:

1. Punctuation, both sufficient and accurate

2. Spelling

3. Pronoun agreement (case, number, person, type)

4. Verb agreement (tense, number)

5. Numbers (accuracy, word versus figure)

Editing/Revising for Style :

This type of editing/revising is done by reading phrases and sentences for clarity, tone, readability, consistency and other qualities which contribute to overall skillful writing. The goal of this process is smooth, clear, unambiguous papers which can be read and understood in one reading.In editing/revising for style, check for the following:

1. Word precision

2. Wordiness

3. Omitted words

4. Nominalization: use of nouns where a verb would be better

5. Technicality of words for audience

6. Tone and level of formality of words -- consistency

7. Redundancy

8. Sentence patterns

9. Sentence length

10. Sentence openings

11. Passives: use of passive where active would be better

Editing/Revising for Structure :

This type of editing/revising is done by reading through the entire document to discern a structure, its relevant parts, and their relationship to the whole. The goal of this process is coherence and a sense of direction in the entire paper.In editing/revising for structure, consider the following:

1. Try to formulate a title with 10 words or less. Some granting agencies specify a title with less than 60 letters or characters.

2. Use as short but as descriptive a working title as possible, for your own early reference. Even a couple of words will do.

3. Use a clear adjective-noun combination to identify the project with its generic class.

Example: "Visual Acuity in Infants", rather than "Studies on the Development of Objective Techniques for Monitoring the Development of Visual Acuity in Infants".

4. If necessary to further distinguish the focus of the problem, use a subtitle.

Example:

"Visual Acuity in Infants: Objective Monitoring of its Development"

5. Select words which accent the main category of the study and which help to describe its distinctive features. This is the traditional genus-species method of describing an item.

6. Avoid such fillers and non-communicating devices such as

A Study of... An Exploratory Study to Determine...

An Examination of... A Method to Explore...

unless the focus of your project is the methodology itself, rather than the results of using the methodology.

7. Study titles of other funded projects in your field, for several reasons. (Refer to Smithsonian Scientific Exchange of Information for details of currently funded research projects.) You will get some sense of the type of research currently funded and you will see how specifically other researchers describe their projects. You will also see the extent of precise technical language in your discipline.

8. State the major idea as quickly as possible, with the modifiers following, rather than preceding, the main category.

9. Avoid jargon or vogue words, even though you may use them daily in practicing your profession. Grants are permanent records, accessible to the public, if awarded by a government agency. You want to remain clear, unencumbered by dated or limited language. For example, "parametrize", "infrastructure", "heuristic", "impact" (as verb), "cost out", and "resource utilization" are all being used now in various disciplines. Like most jargon, these words have clear, simple substitutes which convey the same message with more economy and precision.

PROPOSAL WRITING: ABSTRACTS

Project Abstracts: Guidelines

Contents : While the abstract's contents should vary to include information most salient to each project, each abstract should say something about each of the following topics:

Subject:What is the project about?

Purposes and Significance:Why is the project being done? What is to be accomplished? Why is it important?

Activities:What will be done? What methods will be used?

Target Population:What special group is being studied or served?

Location:Where is the work being performed (if different from the location of the Principal Investigator's organization)?

Expected outcomes:What types of findings or results will be produced? To whom will these be useful? How will they advance knowledge or the state of the art in your profession?

Writing the Abstract : In general, abstracts should not exceed 200 words. They are intended to provide a general understanding of what the project is about, not detailed explanations. Each sentence should say something specific and worth knowing.

Allowing 200 words, with roughly 20 words per sentence, the abstract will have approximately 10 sentences. It needs no introductory or concluding sentence and it is generally one paragraph. It may, however, be divided into paragraphs if the topic allows.

The abstract should be written last, after the entire proposal is composed. It should be a very clear, direct statement of the project so that a reviewer can decide whether or not it fits the funding priorities of the agency and who to use as a technical reviewer. Often the abstract is entered into computerized retrieval sources as the project description, so it is worth taking time to write it well.

PROPOSAL WRITING: ABSTRACTS

Types of Abstracts

In technical writing, including proposal writing, there are two basic types of abstracts which may be used: Descriptive and Summary . Most research proposals ask for a Summary abstract, while many journals require a Descriptive one.

Summary : The abstract summarizes the main findings or theories of the proposal or article. The reader can see the projected thrust of the article or project and understands the (potential) outcomes.

Example: Two principal themes are observed in software development, both aimed at improving the productivity of developing and maintaining new applications. The first is to provide increasingly rich system programming function in order to handle the details of managing hardware resources. The second is to provide application development facilities with logical structures and building blocks more closely aligned with the logic of the application itself. An additional challenge is to provide these in a way that will allow continued enhancement of existing software.

Descriptive : The abstract tells what is in the article or proposal, but it does not offer any conclusions or information about the findings.

Example: Enormous progress in electronic technology is accelerating the use of computers in everyday life. In this article trends in hardware, input-output technology, computer architecture, software, communications, and artificial intelligence are examined and complexity is identified as a limitation to further progress. Promising directions of research, which may extend the range of computer applications, are discussed .

(Additional Samples on page 21)

CHECKLIST FOR STATEMENT OF GOALS, OBJECTIVES AND ACTIVITIES

1. Are there one or more goals that reflect the need for the project and clearly show its purpose and direction?

2. Is there at least one objective for each need or problem committed to in the needs/problem statement?

3. Are the objectives realistic and appropriate?

4. Are the objectives stated in terms of outcomes and not in terms of methods or activities?

5. Do the objectives describe the population that will benefit?

6. Do the objectives state the time by which they will be accomplished?

7. Do the objectives describe the outcomes of the project in measurable terms?

8. Do the activity statements show how the objective will be put into operation and accomplished?

AESTHETICS CHECKLIST

1. Does the cover express something of the content of this proposal or the nature of your organization?

2. Is there a Table of Contents?

3. Is there a title page with all necessary background information describing this document?

4. Is the summary no more than 3/4 of a page long?

5. Does the organization of the proposal enhance the content and make it easy to find/avoid types of information?

The proposal is received and numbered by the reception center or application control center of the federal agency. Within 30 days, a card should be received by the project director indicating the number that has been assigned to the proposal. If this card is not received in 30 days, contact the program manager.

Upon receiving the proposal, the program manager or staff member will conduct a preliminary review for the required components of the proposal. If these components are not included, the proposal will be sent back and the project director will be requested to supply the necessary information.

If the proposal is complete, the review process commences. The process can entail three separate review procedures and generally consists of at least two of the three methods presented below.

1. External Panel Review . A copy of the proposal is sent to reviewers located around the United States who are experts in the discipline. If a proposal is in a specialized area, it is appropriate to submit a list of potential reviewers. The agency is under no obligation to use these; but in most cases, one, if not more, of the potential reviewers will be asked to review the proposal.

2. Panel Review . The panel is a chosen group of experts within the discipline(s) who meet on a regular basis and rate proposals submitted to a particular program. The list of review panelists is public information and can be obtained from the agency.

The proposals are sent to the reviewers before the panel convenes. During the panel sessions, proposals are rated in rounds. In most cases a narrative review will accompany these point-rating systems. The program's staff has input during these sessions, but it is primarily concerning technical information.

3. Program Manager and Agency Staff Review . This review is separate from the external mail review and the panel review. The program manager usually follows the comments and ratings of the reviewers, but a program manager has the authority to reject or approve a proposal based on his/her own judgment and knowledge of external factors.

A proposal may be withdrawn from the reviewing process by the project director at any time. If withdrawal is initiated by the project director, a letter of withdrawal must be written by the project director, and a copy sent to the appropriate institutional office.

In certain agencies such as the National Institutes of Health (NIH), the second round of reviews is made by an advisory council which determines funding priorities.

FEDERAL AGENCY REVIEW PROCEDURES, cont.

B. Request for Reviews

The project director may request copies of the anonymous written reviews from most agencies. Inquire about the procedures of the granting agency for obtaining reviews. These reviews are useful for all project directors, even if the proposal is granted an award. The reviews can be used when conducting the research or when revising a proposal for resubmission at a later date.

C. Budget Negotiations

If a project director is contacted by the program manager to negotiate the budget and scope of the project, inform the institutional representative. Any significant changes in the budget or the scope of the project must be made through and approved by the institutional representative.

D. Award or Declination

The award letter or declination notice will be sent by mail. In general, agencies are restrained from making verbal announcements of awards or rejections. The award letter must be received prior to spending any funds. After receiving either an award or declination letter, notify the institutional representative. ORTTA will review the award and establish an account, assuming the appropriate submission procedures have been followed.

COMMON REJECTION REASONS

A few years ago NIH analyzed the reasons why over 700 research proposal applications were denied. Their findings are worth reviewing before preparing a proposal for submission to NIH or any other granting agency.

I. Nature of the Problem (18%)

A. It is doubtful that new or useful information will result from the project (14%).

B. The basic hypothesis is unsound (3.5%).

C. The proposed research is scientifically premature due to the present inadequacy of supporting knowledge (0.6%).

II. Approach to the Problem (38.9%)

A. The research plan is nebulous, diffuse and not presented in concrete detail (8.6%).

B. The planned research is not adequately controlled (3.7%).

C. Greater care in planning is needed (25.2%).

1. The research plan has not been carefully designed (11.8%).

2. The proposed methods will not yield accurate results (8.8%).

3. The procedures to be used should be spelled out in more

detail (4.6%).

D. A more thorough statistical treatment is needed (0.7%).

E. The proposed tests require more individual subjects than the

number given (0.7%).

III. Competence of the Investigators (38.2%)

A. The applicants need to acquire greater familiarity with the

pertinent literature (7.2%).

B. The problems to be investigated are more complex than the

applicants realize (10.5%).

C. The applicants propose to enter an area of research for which

they are not adequately trained (12.8%).

D. The principal investigator intends to give actual responsibility

for the direction of a complex project to an inexperienced

co-investigator (0.9%).

E. The reviewers do not have sufficient confidence in the applicants

to approve the present application, largely based on the past

efforts of the applicants (6.8%).

IV. Conditions of the Research Environment (4.8%)

A. The investigators will be required to devote too much time to

teaching or other non-research duties (0.9%).

B. Better liaison is needed with colleagues in collateral

disciplines (0.4%).

C. Requested expansion on continuation of a currently supported

research project would result in failure to achieve the main

goal of the work (3.5%).

Based on the above analysis, a carefully designed, well reasoned proposal will overcome these common pitfalls. It also represents and important credibility statement about the investigator.

------------------

A comparable study was conducted in the Bureau of Occupational and Vocational Education. Based on a sample of 353 research grant applications,

-- 18% forgot to number the pages.

-- 73% forgot to include a table of contents.

-- 81% had no abstract.

-- 92% failed to provide resumes of proposed consultants.

-- 25% had no resume for the principal investigator.

-- 66% included no plan for project evaluation.

-- 17% forgot to identify the project director by name.

-- 20% failed to list the objectives of the project.

All proposals should be double-checked for these and similar weaknesses. A few minutes spent in careful proofreading can catch these problems before the reviewers do.

CONSIDERATIONS IN OBTAINING GRANT AND FOUNDATION DOLLARS

(Major reasons for being turned down)

1. Must not ask for just operating funds. Funding agencies are looking for tangible signs of effective working concerns relative to major societal issues.

2. Agencies must believe that their money will be making a significant difference in the areas of policy, procedures, or services as a result of the project. They are also interested in how their funds will change the responsiveness of the institution to the needs of its clients and the community. These issues must be addressed in all proposals.

3. The funding organization will want evidence that the organizational board has shown an attempt to develop a leadership role directed toward a continuing process of adaptation to changing societal needs.

4. The funding organization is concerned if you treat their funds in the same way that you do tax support. Their analysis is that they might as well pay more taxes and let you obtain more money from the government rather than receive their funds. The point is that you must do something unique with funds from non-governmental sources.

5. Funding organizations will look hard at the evidence you present in your proposal for its integrity and application to the problems and issues at hand. There also needs to be a solid evaluation plan. The evaluation plan must not only monitor on-going activities but assess whether the funds are making significant differences regarding services.

6. It is important to provide testimony from grant groups to be served in addition to your own analysis of their needs. A critical criteria along this line is that your project encourages clients to be less dependent on your agency rather than more dependent over time. The goal appears to be increased self-reliance rather than increased dependency upon institutional assistance.

HELPFUL SUGGESTIONS FOR PREPARING PROPOSALS

FOR EXTRAMURAL SUPPORT 1

Recent conversations with program officers at the National Science Foundation and at the National Institute of Health produced ideas which may be generally useful in the preparation of applications for extramural support of research projects or scholarly activities:

How Will Your Proposal Be Judged?

In preparing a research proposal it is essential to bear in mind that it will be read and judged by astute reviewers who are experts in your specialty. Many other authors will be competing with you for the attention and favorable reaction of those reviewers. It may be useful to consider some of the kinds of questions that will arise in the minds of the reviewers as they evaluate your proposal and determine its merit ranking relative to other competing proposals.

1. What exactly is the work to be accomplished ? The proposal should be as explicit on this point as is possible.

2. What is the current state of our knowledge ? Where does one begin in working on the task to be accomplished? The author should assume that reviewers will be as well informed on this point as he or she is. They will almost surely react unfavorably if the author appears to be ill-informed on this point.

3. What will be the state of our knowledge after successful completion of the proposed work? To answer this question the proposal must communicate:

a. The criteria for determining when the objective has been attained.

b. The nature of the boundary conditions surrounding the state of our knowledge when work toward the objective began.

c. The criteria by which progress toward the goal may be measured.

4. What are the methods of procedure for performing the proposed work? Is it clear that these methods are applicable to the proposed task? Are they independent and non-sequential? Is each method subject to independent verification? Is it possible to estimate the probability that the chosen methods will, in fact, yield the desired information? Have all the apparent methods been considered and from those, have the most promising been selected? Are the limitations of the chosen methods recognized? Does the author of this proposal have the expertise which the methodology requires?

5. Is it possible to predict the consequences that will fall out from achievement of the stated objective? Are the positive, as well as negative, value functions of these consequences recognized?

6. What are the potential benefits to be derived from achieving the stated goal?

Matchett, cont.

7. What is the probability of successful achievement of the goal? Is "success" subject to an "all or none" effect or is it possible that the work could be partially successful?

8. How much effort will be required to perform the proposed work? How much support of all types will be required to perform the work? Is the author's estimate of the required effort realistic?

9. Taking into account the answers to questions 5, 6, 7, and 8, and considering the severe limitation on available funding, why should this project be supported ?

Why Are Proposals Turned Down?

One of the most helpful publications on this question is an article written by Ernest M. Allen, "Why Are Research Applications Disapproved?" ( Science 132:1532-1534, 1960). This study, even though twenty-one years old, is still contemporary and relevant. (Cf. Eaves, G.N. "Who Reads Your Project-Grant Application to the National Institutes of Health?" Fed. Proc.31:2-9, 1972) Table I of this article lists several classes of deficiencies which tend to recur in research applications which fail to receive approval. These deficiencies may not all be relevant to your particular work, but they can serve as a rough checklist of mistakes to avoid in preparing an application. They are:

1. The problem is trivial or is unlikely to produce new or useful information.

2. The proposed research is based on a hypothesis that rests on doubtful, unsound or insufficient evidence.

3. The problem is more complex than the author realizes.

4. The problem is local in significance, production, or control, or otherwise fails to fall clearly in the mainstream of the discipline.

5. The problem is intellectually premature - only a pilot study.

6. The problem as proposed is overly involved with too many elements required to be investigated simultaneously.

7. The description of the research leaves the proposal nebulous, diffuse, and without a clear aim.

8. The proposed methodology, including tests and procedures, are unsuited to the objective. May be beyond the competence of the investigator.

9. The over-all design is not carefully thought out.

10. Statistical aspects are not given sufficient consideration.

11. Approach lacks imagination or originality.

Matchett, cont.

12. Controls are either inadequately conceived or described.

13. Proposed material for research is unsuited or difficult to obtain.

14. The number of observations proposed is unsuitable.

15. Available equipment is unsuited to the research.

16. Investigator does not have experience or training for the proposed research.

17. Investigator appears to be unfamiliar with pertinent literature or methods, or both.

18. Investigator's previously published work in the field does not inspire confidence.

21. Investigator needs more contact with colleagues in this or related fields.

22. Requirements for equipment, personnel or time are unrealistic.

23. Other responsibilities prevent the investigator from devoting sufficient time to this project.

24. Institutional setting unfavorable.

25. Current research grants held are adequate in scope and funding to cover the proposed research.

In general, disapproval of the request results from a combination of these faults in the eyes of one or more of the reviewers. Probably the largest single group of faults would come under the heading of "Methodology", or in the investigator's ability to carry out the research in the manner proposed.

Trivial Errors That Can Shoot You Down

A memorandum from a major section of the National Science Foundation states "Many proposals received...present incomplete data which results in processing delays." The data most often missing were biographical material for the principal investigator; listing of all current research support including title, source, period and amount, and fraction of effort devoted to the work; list of all pending proposals; for renewal proposals, expenditures under current grant by major categories and a budget for the uncommitted balance; and a new budget for each year for which support is requested and a total budget page. The importance of including a 200 word abstract of the proposed work is also stressed. In other words, try to follow requested formats very closely.

Matchett, cont.

The Need For Peer Review

Perhaps the single most important factor in preparing a successful application is peer review prior to submission to the agency. The University requires administrative review by the department chairperson and the administrative unit prior to submission. There is no University requirement for peer review, yet in the long run, it can be the most important of all. Such review is necessarily left to the investigator, since he or she is best able to identify reviewers with appropriate competence and interest.

What To Do If You Are Turned Down

If your proposal is not accepted by a sponsoring agency, one of the most important things you can do is to request evaluations and reviewer comments. Not only will these evaluations indicate the weak points of your project, but they will be invaluable in submission of future proposals. The success rate with most federal agencies is less than 25%. The best way to get your projects funded is to use the reviewer comments and apply again.

Other Suggestions

One way to keep current on topics of interest to sponsoring agencies is to volunteer to serve on review panels. Also, when considering a sabbatical or leave of absence, you may want to consider working for a federal agency closely allied to your research areas. This would also help you to know of current research emphases.

Excerpts From an HHS Guide for Evaluation of RFP's

These excerpts give specific points for the reviewers to look for. It should be helpful to anyone seeking external support of projects.

Scope of Work

This section of the proposal should be the most comprehensive because it reveals the offeror's knowledge of the field and contains the suggested approach for performing the requirements of the evaluation study. In most instances, the heaviest weight will be given to this section of the proposal.

Has the offeror demonstrated adequate knowledge about the background, operations, and status of the program to be evaluated?

Has the offeror presented an approach which will achieve the stated objectives of the RFP?

Is the proposed approach supported with justification of why it should achieve the evaluation objectives?

Do you think the suggested approach will work?

Has the offeror introduced unanticipated events which may result in a project overrun or an expanded scope of work?

Has a specific management plan by task for period of performance been included?

Has the offeror demonstrated efficient use of time and resources, especially if special services such as computer time are required for a short duration of the study?

Has the offeror been realistic in the amount of time allotted for the performance of each task?

Has the offeror demonstrated competence in a highly specialized area, such as statistical analysis, which is required for the evaluation study?

Has the offeror allowed for slippage in the preparation of questionnaires, test instruments, test administration, data process, etc.?

If appropriate, have site visits been adequately provided for throughout the period of performance?

Are reports keyed to major milestones/events of the study?

If appropriate, has the offeror provided for use of community resources?

If data collection is required for a comparative study, has the offeror allowed for an adequate sample of an experimental or control group?

Does the offeror specify the products which will result from the evaluation study?

Has the offeror allowed for OMB clearance on the development of measurement instruments?

HHS Guide , cont.

Has the offeror demonstrated knowledge about evaluation techniques and procedures?

If appropriate, has the offeror indicated that an adequate representative of all levels of program personnel will be included in the evaluation?

These questions are not all inclusive and items specifically related to the proposed study should be added.

Personnel

Proposed personnel should be examined critically because they are critical to the successful completion of a study. The capabilities, experience, and training of the personnel relative to their specific assignment on the study should be explicitly reviewed. If the RFP also requested references for projects on which personnel worked, then these references should be at least spot-checked.

Is it clear to which tasks in the study specific personnel will be assigned and for what length of time?

Are the personnel assigned to specific tasks qualified by training and experience to successfully perform the tasks?

Has enough information been provided about personnel to allow adequate judgments to be made about their proposed roles in the study?

Is the apportionment of personnel level and time to specific tasks realistic?

What assurances are made concerning the availability of personnel proposed? Was a contingency plan requested if certain personnel become unavailable?

Have enough time and personnel been included to provide adequate administrative management of the study?

Are consultants to be utilized; if so, to what extent? Is the proposed use appropriate?

Is the author of the proposal one of the key personnel?

Does the success of the project depend, to a large degree, upon personnel not directly associated with the prospective firm?

Facilities

On-site availability of special facilities or easy access of required facilities must be indicated in the proposal. The source of facilities and equipment necessary for successful completion of the study, but which is not on site, should be stated, as well as the expected provision for use.

Are the facilities and equipment needed for successful completion of the study specified in the proposal?

How does the offeror intend to access facilities not at the contractor's site?

HHS Guide , cont.

Does the use of facilities outside of the contractor's firm require a subcontract?

If subcontracting is necessary, is the proposed subcontractor specifically mentioned, along with an explanation of its required qualification?

Is the planned use of facilities such as printing, data processing, etc., realistic in terms of the planned evaluation?

Is a realistic time schedule planned if some services are to be performed at facilities located apart from the contractor?

If computer services are required, are there controls built into the processing so corrective action can be taken at intermittent points if necessary?

Past Performance

An organization's "track record" supplies some insight into the firm's capability to perform activities within specialized areas. Reference to past experience establishes a frame of reference from which to judge organizational capability. However, it can also be misleading in terms of the requirements of a study. Keep this in mind during the evaluation process. Glossy, vaguely worded statements with little support provide meaningless information. If the proposer has been asked to provide references regarding work performance, then the contracting officer should spot-check those given.

Do the references to past experience include activities specifically related to the requirements of the proposed study?

What reputation does the firm hold in the area of the proposed study?

Has the proposer been honored by professional societies because of their performance in a specific professional area?

Are the statements of past performance worded in a meaningful way so you can identify what work was actually performed?

Has the offeror bid for a contract in an area where the performance has not yet been demonstrated?

One or more meetings of the evaluation panel is held to determine the acceptability, unacceptability, or potential acceptability of the technical proposals. Each proposal must receive an absolute, rather than a relative, judgment; a pre-determined cut-off score must not be used. A proposal is considered acceptable if, without qualification or revision, the panel judges that the offeror can perform the work competently. An unacceptable proposal might be made acceptable with the submission of clarifying data and therefore be included in the zone of consideration. This may delay the final award for a few weeks. An unacceptable proposal requiring major revision would not receive a rating worthy of selection for the competitive zone.

HHS Guide , cont.

The contract file must contain documentation of who and why certain decisions were made in the evaluation of proposals. The responsibility for this justification rests with the technical representatives on the panel. These representatives rate and rank each proposal on a separate score sheet, then state why those ratings and rankings were given. This is especially critical if a debriefing is requested by contractors who want an explanation of their proposal's deficiencies. Rarely will contractors pursue this issue beyond the normal debriefing unless they feel unfairly treated, discriminated against, etc. Evaluations of proposals should be carefully thought out and recorded if the unhappy occasion should require presentation of this evidence in court.

Cost Information

(Note: The evaluation panel reviews cost information after considering the technical aspects of the proposals. The responsibility for evaluation of costs often rests primarily with the contracting officer, who relies on input from other members of the evaluation panel.)

Is the overall cost within the rate of your (the contracting agency's) budget?

What is the relationship between the cost figures and equivalent items in the technical proposal?

Are the personnel costs reasonable according to the tasks to be performed?

Are the appropriate personnel assigned to perform the appropriate tasks?

Have expenditures been set aside for subcontracting requirements such as data processing?

If a large-scale questionnaire must be mailed, has an adequate sum been set aside for postage?

Have costs for development of instruments, purchase of materials, such as scoring sheets, etc., been included?

Does the travel seem reasonable when compared to the tasks to be accomplished?

If consultants or experts are included, is their daily rate reasonable and within the proper financial range for your agency? Is the proposed time reasonable?