Check Study Division

INTRODUCTION:

GRA Check Study Division:

The GRA Check Study Division (CSD) is an innovator in the use of negotiable checks to conduct a variety of specialized research programs. The CSD uses negotiable checks to complete highly quantitative primary research studies such as "Awareness/Trial/Usage" studies or studies of promotional activities using tightly defined test and control groups. In addition, we have a massive normative database (shown below) of readership studies: APX (Ad Page Exposure) covering multi-sponsored journals, single-sponsored journals and all forms of direct mail. Further, the check study methodology can be used for profiling audiences for promotional purposes, administration of "programmed learning" programs and many other unique concepts.

Primary Research/ATU:

Check studies can be used for any type of primary research (such as the Awareness/Trial/Usage studies) when the number of questions required is limited and the questionnaire can be developed using closed-ended questions or very simple open-ended questions. Response rates are usually significantly higher than any other response-generating protocol. Check studies are very cost-competitive.

Promotional Evaluations (Pre-Post Studies):

Check studies are a powerful tool for evaluating the effects of promotional efforts. Carefully controlled samples can be studied on a pre/post basis (test/control) to determine reaction to promotional stimuli.

Audience Profiling:

Audience behavioral characteristics/practice profile data can be generated to identity high profile physicians/respondents and key marketing candidates. This can be crucial for products that are not adequately measured by standard syndicated prescribing audits. Data can be reported on a respondent-by-respondent basis, tied to sales territory configurations.

Programmed Learning Programs:

Promotional messages can be delivered through the check study programs. Checks may accompany product samples or promotional materials to provide an opportunity for recipients to respond/comment about the materials.

Readership (Ad Page Exposure):

Readership (APX - Ad Page Exposure) Studies measure the share of audience members that sees test materials. It measures the ability of the vehicle (direct mail or a journal) to deliver "eyes to the page". GRA has developed a massive normative database of readership studies covering multi- and single-sponsored journals and all forms of direct mail.

APX Details

For detailed information & protocol for conducting APX Studies

For access to the full GRA APX Normative Database

A FRESH LOOK AT APX STANDARDS

Back in the mid-70’s, the Pharmaceutical Market Research Group (then the Eastern Pharmaceutical Market Research Group) addressed the issue of setting standards for advertising page exposure studies of medical journals. Prior to that time, studies were generally conducted by publishers to measure readership of their journals using a variety of different methodologies. Unfortunately, the differences in methodology…rather than true differences in readership…could often influence study results. Needless to say, some publishers were motivated to “load the methodology” in their own favor and advertisers were ill-equipped to accurately evaluate their results. It was clear that a set of standards was needed.

The need for industry-accepted standards still exists. But the industry has changed and the old standards are in need of an update. It is time to review the role of the APX technique, the new uses for APX and look at how the rules should be implemented today.

WHAT IS APX? WHAT DOES IT MEASURE?

APX is a measurement of the likelihood that a targeted reader will actually see materials mailed to him/her. It tells us how good a job the publisher – or designer of a promotional piece – has done in attracting the reader to the mailing. It measures "eyes on the page" - it does not measure the “impact” of the materials.

HOW DOES APX WORK?

The process is simple. Negotiable checks are place within a journal or within the pages of any type of mailing. The checks are carefully counted and coded to identify the specific page or location on which the check was inserted. The check is a two-part form. Half is the actual check and the other half (which is perforated and torn off by the physician) carries a brief explanation of the purpose, so that the reader will know it is a real check. When endorsed and cashed, the check provides a place for the reader to indicate if he/she found the check personally (self-exposure). Cashed checks are counted to provide a measure of readership…what share of target readers found the check and cashed it?

WHAT CAN BE MEASURED?

When the initial APX standards were prepared, they were directed toward “open” publications or journals…that is, publications with multiple advertisers. The studies were conducted to provide support for advertising expenditures and journal selection. Today, it is widely recognized that various media compete for the doctors’ and other healthcare professionals’ eye. This includes standard multi-sponsored journals, single-sponsored publications, direct mail advertising, etc.

WHY DO WE NEED TO REVISE THE STANDARDS?

The original “standards” were prepared as a guideline for testing multi-advertiser publications. It was assumed that testing would be initiated by the publisher and that the results would be used for promotional purposes by the publisher. For studies of this type, many of the basic rules still apply.

However, the APX technique is now used to study other vehicles. Single-sponsored publications are, perhaps, the most frequently tested by testing is also conducted to measure receipt of the promotional flat mail and “high-impact” mailers such as audio/video cassettes, etc. In fact, the technique can be applied to measure receipt of nearly anything a marketer may send to members of the target audience. Today, not all of the “standards” apply. Some revision and adaptation is necessary. The following comments are not meant to replace the PMRG published standards. They are offered first as a review of those standards…since most of us have long ago lost our copies. Secondly, they are offered to provide commentary and suggestions from one who has conducted extensive APX testing since the late 70's.

CURRENT PMRG STANDARDS & COMMENTS

I. STUDY DEFINITION

PMRG: In order to establish page exposure for a journal, a minimum of three issues should be studied. A journal should be restudied yearly.

COMMENTS: The idea of studying three consecutive issues of a multi-source journal and “averaging” the results remains valid and preferred. It is rarely done, however, due to the cost of conducting three studies and the time required. For a monthly publication, the total elapsed test time through final report would be about 6-7 months. This often conflicts with media planning requirements.

Often, single-sponsored publications or other “one-shot” mailings lack sufficient frequency. Testing three issues is simply not practical. Testing is usually conducted on a one-time basis and independent normative data are required for comparative analytical purposes. An annual “restudy” is still advisable. Conditions do change.

PMRG: A new journal/publication may be studied anytime after publishing begins. However, it’s scores in the first six months of publication should be noted as “new book” scores. In order to be compared to an established journal, a new book should be out for a minimum of six months. Scores from the first six months of a new journal’s issues should not be mixed with later scores.

COMMENTS: Scores for a “new publication” should, indeed, be noted. The theory goes that it takes a new publication at least six months (probably longer) to become “established” and reach it’s appropriate level of readership. As such, initial scores may be low. It is interesting to note, however, the GRA database indicates marginally higher performance for new versus established publications. This may be the result of initial curiosity that attracts primary evaluation of the publication.

PMRG: A publisher may perform his own APX study, provided that care is taken to ensure that the names of the doctors in the sample are not available to the publisher (a letter from the mailing house attesting to this is suggested). If a publisher performs his own study, at least one industry observer must be present for the testing of each issue.

COMMENTS: The rule holds – the publisher must not have access to the list. From a practical point of view, this means that the publisher cannot physically perform the preparation of checks or insertion into test issues. A third party must be involved.

II. PRE-ANNOUNCEMENT

PMRG: All published APX studies must be pre-announced. Announcements can consist of letters to manufacturers and agencies, and/or announcements in appropriate industry publications. Each specific issue to be tested should be pre-announced. If several issues are to be tested, they may all be pre-announced at the same time.

COMMENTS: Tests of all multi-sponsored publications which are conducted by a publisher in support of space sales must be pre-announced. A test of this type should include three issues. The three issue test is considered as one study which requires only one announcement. Tests of single-sponsored publications or any other “privately sponsored” activity are not usually announced to the industry unless some form of public sponsorship is to be solicited.

III. THE SAMPLE

PMRG: Samples can be of any size, provided that the 90% confidence levels for any score appear immediately after that score in the final report.

COMMENTS: Sample size is flexible. Studies typically cover a mailing of at least 400-500 names, which is adequate to provide desired statistical significance. More check insertions may be required in segments are stratified for special observation.

PMRG: Samples can be drawn from the full circulation list, or can be stratified by specialty, etc.

COMMENTS: Stratified segments may be selected and reported individually. However, proper attention should be paid to “totals” which should be weighted if stratified subgroups are tested.

PMRG: The sample may be selected in either of two ways:

A single sample for all three issues can be drawn at one time provided that the sample for a given issue is checked against the circulation for that issue at the time of testing.

A separate samples can be drawn for each issue being studied.

Any method which yields a random sample, such as Nth name selection, or random numbers drawn by computer may be used.

COMMENTS: When three issues are tested, the samples must be checked very carefully. Instructions to the programmer should be prepared, in writing, before any work is done on the project. Plans must be made for list selection for all test issues and the remaining full circulation prior to the first mailing.

PMRG: In order to prevent issue-to-issue sample duplication, each list of doctors to receive checks must be 100% checked, manually, against the sample for other issues to be tested. In addition, each sample for an issue must be 100% checked manually against the full circulation of that issue to make sure that no doctor received two copies of a journal.

COMMENTS: Due to potential for error in programming, the three samples should be manually checked to ensure there is no duplication. In addition, each test sample must be manually checked against the full circulation to ensure mutual exclusivity.

Once it is ascertained that the program is working correctly, it is unnecessary to check 100% of the names. Rather, a substantial number of randomly selected names should be checked. These names should be selected throughout the list.

IV. THE CHECK

PMRG: A description of the check is as follows:

The check will be a double voucher check, on bank green 20lb. long grain stock. All printing on the check will be in black ink. The wording on the check and voucher, both front and back, should follow exactly the sample copy (available from GRA).

For A and B size journals, the check should extend to 1/8 to 3/8 of an inch from the side edge of the page.

Checks should be for $5.00, and should be made out in the doctor’s name with a black felt-tipped pen.

No additional information, other than the standard questions (to ascertain "self-exposure") should appear on the back of the check.

COMMENTS: The rules hold. Availability of paper supplies and printing contingencies may affect the exact size and shape of the check slightly. When results of any study are compared to any normative database, it should be verified that all studies in that base were conducted with exactly the same type of checks.

In practice, most clients want to include some questions on the back of the check in addition to the “standard questions” required to establish “self-exposure”. These questions should be kept brief and closed-ended. Typically, over 95% of respondents answer these questions. There is no evidence to suggest that a few additional questions on the back of the check have any negative effect on response to the study.

V. EXECUTION

PMRG: The check will not be attached to a blank page, but will be “tipped-in” to the journal. The check should be attached to the page on the gutter side, using “double-stik” type tape, not glue. There will be no glue or tape on the outside edge of the check.

COMMENTS: Agreed. Most specialty check printers can produce checks with a glue strip with a tear-off protector. This is a lot easier to work with than double-stik tape. Checks must be prepared with the option of insertion on either right or left hand pages.

PMRG: The check should be centered on the ad page to be tested. This includes full page ads in A and B size journals and any ad island-half size or larger in tabloids.

COMMENTS: In general, every ad page should be tested. However, sometimes this is not practical. For example, there may be too many ad pages or there may be many fractional pages. There is little reason to affix checks on the full-disclosure page of the unit.

PMRG: Cover positions, inserts and editorial may be tested, but results for each of them should be reported separately.

COMMENTS: To assess true readership of a publication, both ad and editorial pages should be tested. A rough mix of ad/editorial positions may be selected to mirror the ad/editorial ratio of the publication. If this is impractical, added weight is usually given to the advertising positions. Ad and editorial positions should be reported separately.

PMRG: The basic precaution to be taken is that the check does not break the book, does not extend beyond the edge of the page and that the check remains firmly in the book.

COMMENTS: Agreed. The check should not break the book. In addition, when checks are inserted, care should be taken to ensure that the workers do not crease the binding or otherwise damage the publication in a way that will cause it to “fall open” at the test position. This can happen with surprising ease.

VI. TEST COPIES

PMRG: The test copies of the journals must be inserted into the full run of the journal, in exact zip code sequence. Test copies must be bundled along with regular copies and when mailed, should be indistinguishable from regular copies (i.e.: should be a letter from the mail house certifying this was done).

COMMENTS: Due to the extraordinary cost of holding up a production run, it is usually impractical to reinsert test issues back to the full run in exact ZIP sequence. This is almost never done. Rather, test issues are usually mailed in “mixed state bags”. As long as these bags are mailed at the same time as the full circulation, there is no reason to believe there will be any difference in delivery time. The issue will look exactly the same when received.

PMRG: Local postal regulations must, of course, be followed regarding extra postage for the test copies. However, if the backs of the checks are metered, the meter should not be an additional piece of paper, but on the check itself. Postage stamps must not change the the way that the journal issue is mailed (i.e.: it should be mailed at controlled-bulk rate.)

COMMENTS: The researcher must always check with postal authorities. Since the procedure is a “research technique” and not a typical part of the publication mailing, additional postage is not generally required. If it is, caution must be taken to ensure that the postage does not alter the appearance of the test mailing.

VII. REPORT

PMRG: BACKGROUND MATERIAL

The report must include sections on:

Sample selections

A comparison of the sample composition compared to the full circulation

Accounting for every check

Description of insertion procedures

Mailing dates of test issues and comparison with mailing dates of prior issues

COMMENTS: All essential.

PMRG: Reports should include the following information:

Description of the test issue (e.g.: circulation by specialty, page size, frequency, etc.)

Separate scores by issue;

The number of pages in the test issue, and in an average issue of the journal (using at least the last 6 issues).

The ad/editorial ratio for the test issue, and for an average issue of the journal (using at least the last 6 issues).

An actual sample of the check – not a photocopy.

COMMENTS: A detailed description of the test piece should, of course, be included. Since many tests are of single-sponsored publications, the “average number of pages/last 6 issues” and ad/editorial ratios may not be appropriate to report. An actual copy of the check adds a nice touch to the report, but a photocopy provides the same information.

VIII. APX CALCULATION

PMRG: For each APX score reported, checks found by people other than the physician (i.e.: not self-exposed) should be subtracted from the numerator and denominator of the APX ratio. The same applies where the question “Did you find the check yourself?” was not answered. Undeliverables should not be subtracted from the sample size. APX results for covers, inserts and editorial, if tested, should be reported separately. 90% confidence intervals for each reported score, should be included every time that score is mentioned in the report. Cut-off dates (i.e.: the date on the bank statement) for counting checks in the report should be 10 weeks from the mailing of the test issues.

COMMENTS: Agreed. Note, however, that comments relative to undeliverables are not really pertinent because the researcher does not receive undeliverables and has no record of the level (unless a publication is mailed first class). The official standards called for a ten week field period. We have always allowed 12 weeks from the date of mailing of test issues. This provides protection for the publisher in case the publication “mail drop” is delayed at the printer for any reason. From a practical point of view, data are tracked on a weekly basis, so ten week data are available, if desired.

IX. OTHER

PMRG: A statement that the PMRG guidelines were followed should be included in the final report and signed by the publisher. Any industry observer should be named in the report. The observer should send his/her comments, if any, to the president of the PMRG. All records, cancelled checks, etc., should be available for inspection by representatives or manufacturing and advertising agencies upon request.

COMMENTS: These elements should be observed.

CAUTION: A NEW WRINKLE FROM THE BANKING INDUSTRY

Many years ago, a rule was promulgated by the banking industry that prohibits printing on the back of a check lower than one and one-half inches from the top of the check. Technically, this rule makes it impossible to conduct an APX study, since the required “self-exposure” questions would not fit on the back of the check.

Some banks, however, will negotiate an agreement with researchers to permit continued use of special APX checks. Anyone wishing to conduct an APX study should be sure to verify that their vendor has the appropriate clearance. GRA is permitted to print any/all desired questions on the back of the check.

THE APX NORMATIVE DATABASE

The purpose of this database is to provide a structure for reviewing the value of print materials. Norms, by type of vehicle, broken further by specialty and many other subgroups, provide accurate evaluation of readership level and efficiency. In addition, the data provide general direction in a way designed to promote thinking about promotional options in a new light. GRA actually has two different Normative Databases, one for all Publications and another for all types of Direct Mail.

LEGAL NOTICE:

The normative data included in this page are the property of Glickman Research Associates, Inc. These data are provided for use by our clients to establish a parameter for evaluation of current test data.

GRA clients may use this information for sales purposes and copies may be distributed for internal use by any client of Glickman Research Associates, Inc.

The normative data may NOT be published for general distribution or promotional purposes at any time. Current scores for the test publication may be used promotionally with prior written clearance from Glickman Research Associates, Inc.

CAUTION: Since any differences (subtle or overt) in research protocol may influence test results, it is totally inappropriate to compare GRA Normative Data to scores of tests completed by any other vendor.

We thank you for your cooperation.

METHODOLOGY (APX)

All studies are conducted via the “perfect coupon” or check study (APX) method. An Nth sample of names across specialty lines or individual specialties within each mailing is selected from the circulation galley of the material mailed. The names are held aside from the full circulation and are mutually exclusive for each test.

A check for $5.00 is made out to each physician or non-physician on the sample lists. The checks are issued on a special account under the Glickman Research Associates, Inc. name. The checks are prepared to standards established by the Pharmaceutical Marketing Group (PMRG) in terms of monetary value, size and weight of paper.

Test positions are selected to provide coverage throughout the mail piece. The test materials are mailed at the same time as the remainder of the circulation. Twelve weeks are allowed for the checks to be cashed. There are no markings on the package to indicate in any way that test materials are enclosed.

In addition to the inclusion of checks in the mail piece, information is obtained through special questions printed on the back of each check. These questions are answered when respondents endorse the check. Standard questions are included to determine if the respondent found the check personally (self-exposed). A “projected exposure” score is also generated by combining respondents who are self-exposed with those who claimed that although the check was brought to their attention by someone else, they would have ordinarily seen the mail piece. (See "A Fresh Look at APX Standards")

PUBLICATION DATABASE

DESCRIPTION OF PUBLICATION FORMATS:

Single-sponsored Publications – A publication sponsored by one company.

Multi-sponsored Journals – A publication that has more than one advertiser.

GRA APX NORMATIVE DATABASE: PUBLICATIONS

All publications	24.4%
All physician publications	22.8
Single-sponsored	23.2
Multi-sponsored	20.0
All non-physician healthcare professional publications	30.9
Single-sponsored	31.1
Multi-sponsored	24.8


RANGE OF SCORES:
All single-sponsored publications
High score to date	56.6
Low score to date	10.6

All multi-sponsored publications
High score to date	33.8
Low score to date	3.8

GRA APX NORMATIVE DATABASE: PUBLICATIONS

EDITORIAL CONTENT:
All clinical publications	23.8%
All non-clinical publications	26.4
All managed care publications	19.4
All convention guides	27.2

THICKNESS:
All thick publications	21.4
All thin publications	25.7

AGE: *
All established publications	25.3
All new publications	22.9

*Defined as in publication for at least one year

PUBLICATIONS BY SPECIALTIES (alphabetical)

	%
Allergists	32.2
Anesthesiologists	27.6
Cardiac Surgeons	25.3
Cardiologists	18.2
Cardiovascular Surgeons	25.3
CRNAs	31.5
Dentists	34.5
Dermatologists	33.2
Diabetic Educators	27.4
Emergency Medicine	26.1
ENT/Rheumatologists	34.9
Gastroenterologists	24.0
Generalists	20.5
Hematologists/Oncologists	21.5
HIV Treaters	13.1
Internists	16.3
Microbiologists	40.6
Mixed Specialties	23.7
Nephrologists	18.0
Neurologists	21.2
Nuclear Med-Radiologists & Technicians	38.3
Nurses	28.8
Obstetricians-Gynecologists	23.2
Oncologists	17.3
Ophthalmologists	29.4
Oral Surgeons	25.2
Otolaryngologists	32.2
Pediatricians	25.0
Pharmacists	33.2
Hospital Pharmacists	37.5
Retail Pharmacists	30.2
Physician Assistants	35.5
Podiatrists	30.9
Psychiatrists	25.8
Radiologists	27.0
Residents/fellows	22.9
Rheumatologists	30.0
Surgeons	25.2
Urologists	29.0

GRA APX NORMATIVE DATABASE: PUBLICATIONS

(rank order)

	%
Microbiologists	40.6
Nuclear Medicine/Radiologists & Techs	38.3
Physician Assistants	35.5
ENT/Rheumatologists	34.9
Dentists	34.5
Dermatologists	33.2
Pharmacists	33.2
Hospital Pharmacists (37.5%)
Retail Pharmacists (30.2%)
Allergists	32.2
Otolaryngologists	32.2
CRNAs	31.5
Podiatrists	30.9
Rheumatologists	30.0
Ophthalmologists	29.4
Urologists	29.0
Nurses	28.8
Anesthesiologists	27.6
Diabetic Educators	27.4
Radiologists	27.0
Emergency Medicine	26.1
Psychiatrists	25.8
Cardiac Surgeons	25.3
Cardiovascular Surgeons	25.3
Oral Surgeons	25.2
Surgeons	25.2
Pediatricians	25.0
Gastroenterologists	24.0
Mixed Specialists	23.7
Obstetricians/Gynecologists	23.2
Residents/Fellows	22.9
Hematologists/Oncologists	21.5
Neurologists	21.2
Generalists	20.5
Cardiologists	18.2
Nephrologists	18.0
Oncologists	17.3
Internists	16.3
HIV Treaters	13.1

DIRECT MAIL DATABASE:

DESCRIPTION OF MAIL FORMATS:

Personalized letter:	A regular size (mostly #10) envelope with the recipient’s name on the envelope and the letter.

Non-personalized letter:	A regular size envelope with a “generic” title on the enclosed letter and envelope or a regular size envelope with the recipient’s name on the envelope and a “generic” name on the letter.

Brochure/Reprints:	Self-mailers or different size envelopes with pamphlets/folders.

High-Impact mailers:	A three-dimensional piece (often audio/video/samples) with pamphlets/folders.