Act Without a High Quality of Legislative Research

People are upset about a draft bill of the High Quality Research Act (HQRA) proposed by US House Science, Space, and Technology committee Chairman Lamar Smith (R-TX).  But at a glance, it seems like a lot of the responses have overstated the potential impact of such a bill.

The text of the draft bill is here.  The bill has not even been officially introduced so it’s a long way from becoming a law.

There is nothing in this bill that suggests that the decision to award grants would be changed in any particular way.  It seems that all the bill would do is require the Director of the NSF to certify that “the research project

(1) is in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;

(2) is the finest quality, is ground breaking, and answers questions or solves problems that are of utmost importance to society at large; and

(3) is not duplicative of other research projects being funded by the Foundation or other Federal science agencies.

The NSF already updated its grant criteria this year.

Generally speaking, there are two primary criteria that the NSF instructs its reviewers to evaluate, and which it makes its grant applicants describe.  These are called “Intellectual Merit” and “Broader Impact.”  The truth is that there’s often overlap between these, but the NSF has already included the idea of benefit to society as part of its criteria for awarding grants.  they also require that the project be intellectually new and address an un- or under- explored question.  That’s not new.

In January, 2013, the NSF released updated criteria for evaluating the Merit of grant proposals.  Included in the revised criteria are guidelines stating

• All NSF projects should be of the highest quality and have the potential to advance, if not transform, the frontiers of knowledge.

• NSF projects, in the aggregate, should contribute more broadly to achieving societal goals.

• Meaningful assessment and evaluation of NSF funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects.

With that in mind, let’s look at the three requirements that Section 2, subsection (a) obliges the NSF Director to certify.

The first of the NSF’s January criteria is virtually identical to the second requirement of the HQRA.  So really, clause 2 of the HRQA is redundant.

As for clause 3: A truly duplicative project, is virtually impossible to begin with.  Anyone who was proposing to do the exact same thing that someone else had already done is unlikely to be judged to be “advancing knowledge.”  To do something similar but in a different way is not duplicative.  To confirm a result under other circumstances, to refine and add to existing knowledge is not duplicative.  NSF grant applications already have to explain how their project differentiates itself from existing scholarship.

That does NOT mean that every project has to reinvent the wheel. (Actually, just about the only thing you can’t do is reinvent the wheel.)

Clause three could only be applicable if two totally independent groups propose exactly the same project at the same time (and somehow didn’t know about one another enough to collaborate or at least stake out somewhat different approaches.)  If that’s the case, then the NSF is probably already going to decide whether one group is better suited to carry out that research than the other.  The NSF woudl probably also let the two groups know about this duplication and invite them to resubmit modified proposals.  Since most NSF grants are awarded for proposals that have been declined at least once, this is not a real concern.  In a highly competitive grant environment, the NSF isn’t going to award duplicate grants to identical projects.  But again, virtually no two projects are identical.

So what about clause number 1, which appears the most problematic?The NSF lists five “Review elements” that are intend to guide the people making evaluations of grants:

Five Review Elements

The following elements should be considered in the review for both criteria:

1. What is the potential for the proposed activity to: a. advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and b. benefit society or advance desired societal outcomes (Broader Impacts)?

2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?

3. Is the plan for carrying out the proposed activities well-reasoned, well- organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?

4. How well qualified is the individual, team, or institution to conduct the proposed activities?

5. Are there adequate resources available to the PI (either at the home institution or through collaborations) to carry out the proposed activities?

The HQRA mandate to “advance the national health, prosperity, or welfare” of the United States seems to be equivalent to the criterion of “benefit[ing] society or advanc[ing] desired social outcomes?  It might be argued that the HQRA is specifying what those desired social outcomes are, but in practice one might be hard pressed to claim that anything that is judged to “benefit society” is not also “advancing the national welfare.”  (Caveat: I am assuming that “national welfare” does not mean the welfare of the United States as an administrative entity, but rather the welfare of the people of the nation, or the society of those people.)

So what about this national defense phrase? Let’s back up.  The bill doesn’t say that the project actually has to secure the national defense.  It says that the project has to be in the interest of theUnited States, and then specifies some of the various interests of the United States.  The United states is interested in securing national defense.  At first glance, if I’m applying for an NSF grant, how do I show that my project secures the national defense.  Fortunately, the question is answered by the bill itself.  “secure the national defense by promoting the progress of science.”

The bill basically defines the act of promoting the progress of science as being a means of securing the national defense.  That’s a dubious sociological and historical claim.  Is it actually the case that the promotion of science inherently promotes national defense? Sometimes, but that’s not really at issue.  It would seem like a project would have to explicitly oppose national security to fall afoul of this requirement.  But it would be hard to do that and not fall outside the “benefit society” criterion.  If this bill passed, the NSF would not be able to fund this project.

Ultimately, the requirements laid out by the HQRA basically say: The project must be certified to have intellectual merit and have a broader impact.  The NSF already requires this of its grantees.  The NSF already requires summaries of grant proposals, including Intellectual Merit and Broader Impact statements, to be publicly available online.  So really.  This is not changing anything

That doesn’t mean that the bill is completely trivial.  Even though it doesn’t actually change the NSF’s evaluation criteria, it does seem to make a claim of Congressional authority to determine grant criteria, rather than the NSF itself.  It also seems to add another layer of bureaucracy to the grant awarding scheme, which as any applicant can tell you, is already bureaucratic enough.

I feel like this bill is legislative bluster, designed to make people look good by claiming that their opponents voted against “High Quality Research”.  And that IS playing politics with the NSF.  But this doesn’t mean that the NSF is being made to abandon peer review, or fund creationism, or defund climate science, or anything like that.  Seriously, everyone who is saying that the NSF is just going to ratify GOP pseudoscience talking points should go and read the bill.

DISCLOSURE: I am a past NSF Postdoctoral Award Recipient.

Advertisements

One thought on “Act Without a High Quality of Legislative Research

  1. To clarify: I’m analyzing and commenting upon the bill itself, not the substance of Smith’s statements at the public hearing, or to the press. It may be that people are taking his other statements about science research (which are quite objectionable) and projecting them onto the bill.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s