Local in Ann Arbor

The results of the City of Ann Arbor’s Deer Management Project survey and their implications.

Surveys are tricky. There is, literally, a whole science to surveys to make them meaningful and useful.  In designing a survey, one should consider very carefully how the information is to be used.

Is the expressed opinion of the public really going to be used to make a policy decision? (In essence, a referendum.)

Or is the intent only to take the public temperature so that one may gauge the likely outcry over a decision?

Or are you trying to collect real data on the experience of the public that can be used in further policy formulation?

Or – least defensible – is the survey designed as a “push poll”, to convince the public so that a particular policy outcome can be justified by a “public” desire?

A notorious example of this last usage was the Connecting William Street project of the Ann Arbor Downtown Development Authority.  This was designed from the start to keep the usual noisy mob out of the way. (See Section II, “A Flawed Public Process”, in this Library Green report.) In particular, it involved a survey which omitted a downtown park as one of the options for city-owned properties downtown, and downplayed the strong public comment in support of a park.

One reason that surveys should be used cautiously is that the public doesn’t enjoy being deceived, or used.  So it is a good idea ahead of time to know how you really intend to use the results.  Quite truthfully, there are many times when it is NOT a good idea to make a survey into a referendum.  If people think you are asking their opinion, only to dismiss it, they resent this mightily. And yet, public opinion may not point to the best policy outcome.  There are times when political leaders simply have to lead, and referendum results may not acknowledge all the constraints and nuances in a particular issue.

But a survey can be a good tool to assess public reaction, if well-designed and used cautiously.  A good example is the series of surveys conducted by a professional firm on behalf of the Ann Arbor Area Transportation Authority, gauging support for a new county-wide transit millage.  After a number of false starts (we documented this at length, see the Transportation Page), the last version of this plan, with a much reduced service area, was approved by the voters of the City of Ann Arbor, City of Ypsilanti, and Ypsilanti Township.  The success of the millage vote was predicted by the last survey conducted; this enabled the AAATA Board to move ahead with confidence.  Here is the Ann Arbor Chronicle’s summary of those transit survey results.

What was the purpose of the City of Ann Arbor’s survey on deer management?  It was initiated without first providing much information about the problem.  Indeed, despite the report provided to the Council earlier by city staff, this survey seems to be designed in part to determine whether there is a perceived problem.  At times it appears to be constructed to inform the opinions of the public.  At other times, it seems to be a rather aimless fishing expedition.

The Deer Management Project Survey

The City used its survey tool called A2 Open City Hall. The Deer Management Project survey was closed on January 2, 2015. There were 27 questions in all (not numbered, though we’ll be referring to them by number).  A total of 537 responses were recorded.

The questions and summary of responses are no longer available for viewing and the outcome has not yet been posted.  However, if you took the survey, your own responses may still be viewed. See update below.

Like many online surveys, this has a flaw in that it is a self-selected survey sample.  A truly accurate survey of public opinion would be a randomized, carefully balanced sample incorporating suitable geographical and demographic ranges.  But, just like the public vote on a referendum, this counts only the people who care enough to show up.

Such a survey has a potential value in estimating the views of the public on this delicate and difficult subject.  But this survey has many deficiencies.

1. Structural flaws

Some of the questions should have been nested, where one skips questions that don’t apply.  For example, Question 7  asks,

Have your garden plants or landscape been damaged by deer?

Approximately half (51%) of the respondents answered “No.”  But then Questions 8-12 go on to ask about the effectiveness of fencing, repellents, noise-makers and other frightening devices, or use of deer-resistant plants in addressing deer damage.  Obviously half the respondents are not going to have direct experience and their opinions on whether various measures are effective can hardly be reliable.  For useful information, Q7 should have instructed the reader to skip Q 8-12 if the answer was “No”.  This is called use of “skip logic” in surveys and makes results much more meaningful.

2. Asking for expert opinions from non-experts

Questions 14-21 are prefaced by this comment:

Communities who have adopted deer management plans typically utilize a variety of measures to help minimize conflicts with deer and/or to reduce their population.

Support for several of these methods (education about deer-resistant plants, marking deer crossings, managing roadside vegetation, prohibiting supplemental feeding) is queried in a straightforward way. But what is not clear is whether these “non-lethal” methods are to be adopted in exclusion of any lethal method of limiting deer population.  Question 18 does indeed ask about approval of a lethal method, such as use of sharpshooters.  None of these questions asks about effectiveness of the one method being discussed.

Then in Q 19-21 the survey asks for expert opinion:

If the following method were allowed by the DNR, based on your knowledge, do you feel *** would be the most effective non-lethal method in resolving the perceived deer nuisance issues in Ann Arbor?

The three non-lethal methods named are deer contraceptives, trapping and sterilizing deer, and trapping and relocating deer.  There are two very odd things about these questions.

(1)These three approaches were specifically disallowed by the August report from the City Administrator, and it is generally understood that none of them are approved by MDNR at this time.

(2) Even if they were approved methods, how would Ann Arbor residents without expert knowledge be expected to have an opinion on their effectiveness? (Results indicated that most rejected the trapping and relocating option, with 72% either saying “Least effective” or “Don’t know”; for the other two options, about 35% were positively impressed, answering “Most effective” or “Moderately effective”, and 65% negative or didn’t know.)

So what was the purpose of including these questions?  Was it to plant a suggestion in the minds of the public that these might be feasible approaches?

3. Selective omission or emphasis

At the December 10 meeting, the consultant (Charlie Fleetham) showed an inclination to emphasize certain results and also announced the intention to eliminate use of the data gathered for some other questions. For the question,

In general, do you believe deer are a more valued species than other urban mammals (coyote, rabbit, squirrel, skunk, woodchuck, etc.)

he reviewed the results by emphasizing those who had answered “Yes” (at that time, 27.2%) while omitting the point that a heavy majority had said “No”.  (Final results were 139 (26%) Yes and 398 (74%) No.)  There was an audible outcry from the audience, at which point Fleetham looked out and stated rather belligerently, “Well, I think that’s significant.”  In fact, it is rather significant.  It shows that a very small minority of Ann Arbor respondents were willing to give deer a special status among all the diversity of wildlife.  But his emphasis seemed designed to make a different point.

Meanwhile, the last three questions on the survey (which were another example of asking the public for an expert opinion) were phrased in this way:

Research concludes that lethal removal measures are most effective for managing a deer population. Please indicate your level of support for *** within the city.

The three choices were for firearm hunting, bow hunting (both as part of the regular hunting season) or sharpshooters.  (No information has been given as to how such programs would be implemented.)

Apparently the local Humane Society objected to this language.  (Tanya Hilgendorf, its director, is quoted at length by the Ann Arbor News in objecting to the questions.)  It has now been stated that these questions will not be considered in the analysis of the survey.  Now, it is agreed that this was unfortunately worded.  But then, it is on a par with many of the other poorly done sections of the survey.  Will the City simply discard data because of political points brought up by one side of the question?

Useful information

In spite of its many flaws, the survey did yield some useful information.

1. Support for lethal methods

As we stated previously, a major reason for public engagement on this issue is that the use of lethal methods to manage the deer population hits on some real sensitivities.  We do have some answers on this question from the survey.  In addition to the three contested questions at the end, there was a relatively uninflected question earlier, which will presumably be retained:

Please indicate your level of support for using lethal methods such as hunting or sharpshooting to reduce the deer population

Like many questions on the survey, respondents were asked to indicate “strong support”, “moderate support”, or “do not support”.  We have combined the two figures for strong and moderate support in this analysis, since they both indicate an affirmation for the method mentioned.  (All percentages are based on 537, the total number of responses at the end of the survey.)  This table shows results from both the early general question about lethal methods and the specific questions about method.

Note that when asked in general about use of lethal methods, a majority indicated support.  However, there was some uncertainty about specific methods, with “firearm hunting” receiving  little support.  Also to note here is the use of the words “within city”.  Since no details of the method are given, this could be imagined as having hunters generally roaming the neighborhoods, which of course will never happen.  This is really a very strong support for lethal methods, given the lack of specific information.

2. Nature of the concern

Another useful question was

Which of the following concerns do you have about deer in the City of Ann Arbor?

Note that the answers are not mutually exclusive in this case – this was a “check all that apply” question.  A little over 60% of respondents did have some concern.  Here are the answers, ranked.

Clearly the danger of deer vehicle crashes was the largest concern, though very few reported that they or their family members had experienced such a crash (near misses were not counted).  But what is notable here is that a substantial minority reported concern about both garden plants and native vegetation.  A similar percentage were also concerned about the transmission of Lyme disease and other deer tick-borne diseases.  It is interesting that the stated concern about decline in the health of the deer themselves was at nearly the same level.

Where do we go from here?

The survey, flawed as it is, does indicate that there is a perceived problem with the burgeoning deer population in Ann Arbor.  It may be possible to obtain slightly more information if the analysis includes techniques such as comparing an individual respondent’s answers on certain questions.  For example, how does the belief that deer are more worthy than other wildlife influence answers on some other questions?  As of January 31, 2015, no summary analysis has been posted.

The Deer Management Project page on the Ann Arbor city website states that a report will be presented to Council on March 2, 2015.  There are very few indications of what information is being gathered in the interim.  Another public meeting is scheduled for February 5, 2015, 7 p.m. at Slauson Middle School, 1019 W. Washington.  According to the City press release, a review of the survey results will be presented at that time.

Note: Posts on this subject and other resources are listed on our page, What do We Do About the Deer

UPDATE: The survey results are summarized and analyzed in this report on Deer Management Recommendations issued in May 2015.  They break the city into “two cities”, lumping Wards 1 and 2 together, and Wards 3, 4, 5 as a second area supposedly less affected by deer.

UPDATE: The survey summary is available on A2 Open City Hall. Go to Closed Topics and choose the deer project.  Choose the Feedback tab.

Click on the Download PDF hyperlink.  You will have to supply your email address and a pdf will be sent to you in time.  The CSV download is not useful – it is a record of all responses, not summarized.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Explore posts in the same categories: Sustainability

This entry was posted on January 31, 2015 at 3:53 pm and is filed under Sustainability. You can subscribe via RSS 2.0 feed to this post's comments. You can comment below, or link to this permanent URL from your own site.