Why We Test
Several years ago, the Maryland Department of Transportation (MDOT) began using the Transportation Service Human Resources System (TSHRS). Within this system, each business unit recruits, tests, and generates eligibility lists for its unique classifications.
The Maryland Department of Transportation State Highway Administration (MDOT SHA) receives many applications each year for a variety of recruitments. When 26 or more applicants are approved for a particular recruitment, the Recruitment and Examination Division (R&E) must conduct an assessment. Sometimes we conduct written examinations, sometimes we have applicants complete a Qualifications Supplement and sometimes we score the applications. Because no two jobs are exactly alike, we need to use different testing methods to determine which applicants are best suited for each job and which candidates should be referred to the hiring division for an interview.
In every instance, we strive to use a test that will best predict performance on the job. Most applicants do better on a test if they believe that the test is a reasonable way to measure their skills. One’s score may even be higher if the test taker feels that the test provides a fair chance to demonstrate knowledge and skills. Our tests are developed to make them fair and accurate measures of one’s ability to perform well on the job.
Who Writes the Test?
Test writing is a process that involves ongoing partnerships within our organization. Employees who have exceptional knowledge of the classification of the recruitment (Subject Matter Experts – SMEs) assist the Testing Analysts with test development. The SMEs are most often employees who perform well on the job and/or the supervisors to whom incumbents in this position report. Together, the SMEs and our Testing Analysts identify the tasks as well as the knowledge, skills, abilities, and other characteristics (KSAOs) necessary for a person to be successful in that particular job. From those KSAOs, the assessment tool is developed. This tool may include (but is not limited to) scoring points for the number of years of experience in a particular discipline, years of experience supervising, specific education, certification, etc.
What Does My Score Mean?
Test scores are used to group candidates into three categories on an eligibility list – Best Qualified (BQ), Well Qualified (WQ), and Qualified (Q). It is important to remember that scores on one exam should not be compared to scores on another exam; the range of test scores may vary depending on a number of factors such as the difficulty level, type of test, number of items on the test, and number of candidates taking the tests.
In addition, scores on a new edition of a test may be different (higher or lower) than scores on previous editions. With each subsequent administration of an exam, a number of factors may change. For example, while the scoring tool may be very similar to the one used in previous years, the SMEs always have the opportunity to adjust the weights of the various measures. If an area that is strong in your application has been adjusted with a lower weight, your overall score would be lower than last year’s score. Note that the SMEs never have access to any applications and weights are changed only when there is a consensus among the SMEs. Also, each time the recruitment is completed the competition changes. For example, if scores on an examination are converted to the 70%-99.9% range, the rescaled score will show you how you scored compared to others. Your score may be getting lower because the competition is getting higher.
Once candidates are placed on an eligibility list, their score is generally “active” for a period of one year. Occasionally lists are exhausted before this period has expired, and a new test is administered. Eligibility lists can be extended for periods of six months or longer. Candidates should track their status on all eligibility lists that they are featured.
Taking MDOT SHA Tests
First complete your application fully and accurately. Not only does a thoroughly completed application help us approve you and move you into the testing stage quicker, but many of the examinations are ratings of the content of your application. Below are some examples of the different types of tests we administer and some tips that will help you throughout the recruitment process.
Training & Experience Rating of the Application
These evaluations are pretty self-explanatory: with this test, we are rating the information provided in candidates’ applications. Prior to the rating process, approved candidates will receive a letter indicating their status in the recruitment. The letter contains the categories on which the candidates will be rated, as well as a deadline for sending in any supplemental information that may have been omitted from the application. Unlike the Qualification Supplement (see below), the application only gives the categories rather than an actual form to complete. We recommend that candidates complete their application to the fullest to avoid appending any additional information at this stage. See the section on Qualification Supplements below for some additional information on completing your application and providing supplemental information.
Writing Test
Many of our classifications require writing samples. Follow the written directions closely and complete the assignment as necessary. As always, you should review your grammar, spelling, and formatting.
Qualification Supplement
As with the Training & Experience Rating of the Application, here we are looking for experience and training that indicates an applicant is well prepared to perform the duties of a job. Applicants must first show that they meet the minimum qualifications. The information given in response to the Qualification Supplement exam will determine an applicant’s score. There will be instructions within the Qualification Supplement that request that the applicant provide specific information. To efficiently score the large amount of Qualification Supplements we receive, we ask that you do not make reference to an attached resume. Resumes and similar materials often do not include all the information we need and the pertinent information is not always easy to find.
Remember that you must follow the instructions exactly and provide your information in the exact order and format specified in the Qualification Supplement. If, for example, a question asks how many times you have performed a certain task, what the dates were, who your supervisor/employer was, and what your level of responsibility was, be sure to provide all this information each time it is requested. You will not get full credit without these facts. The person scoring your exam will not have time to spend digging out the details from your application form or resume and cannot give you credit for anything that you have not clearly stated. Irrelevant information will not be scored and might make it harder for the person scoring the exam to find the relevant information.
Example
If we ask about your experience driving a dump truck, you might answer, “I drove a five-ton dump truck hauling dirt and gravel, working 40 hours per week for 30 months, from April 1990 to October 1993. My job title was Facility Maintenance Technician III and my employer was RKT Construction Company of Upper Marlboro, Maryland.”
Multiple-Choice Tests
Multiple-choice tests have a strong record for measuring abilities of groups of applicants. Research demonstrates that people who do well on these tests are likely to do well on the job. While the match between test scores and job performance is not perfect, these tests are an effective tool for determining which applicants might be best suited for the job.
When taking a multiple-choice test, skim through the test to see what kinds of problems you have to solve. Do you want to warm up on easy questions? Do you want to work on the hard ones while you are fresh? Do you want to leave some complicated questions until last so that they do not use too much time? Choose the strategy that will work best for you, but be sure to keep track of which questions you need to go back and answer.
A common reason applicants do not score high on tests is that they do not read the questions and instructions carefully. Sometimes we develop information about rules or procedures in the test. These rules and procedures are often difficult and may differ somewhat from rules used in our agency. This information gives other state agency employees and outside applicants an equal chance to demonstrate their skill in solving problems.
Approach a test as if you are performing the duties of the job. Act as if each situation has actually occurred while you are working. Consider all of the information given in the question before choosing an answer and answer the question as it is presented. Do not change the question by assuming other facts about the situation. Think about a good way to solve the problem before you read the answer choices. Consider what you should do as an employee in this position. Can you handle the problem without consulting your supervisor? What level of responsibility do you have? Decide whether the action you choose would cause any problems. Would it solve the immediate problem but allow the same problem to come up later? Is it reasonable and legal? Would any of the other alternatives solve the problem as well without causing new problems? Scratch paper is often an effective tool in finding the correct answer. Use scratch paper to reword the answers in simpler terms that allow easier completion. You may draw a picture or diagram to help you reason out a solution. If you have time left at the end of the testing period, review your answers. If you have good reason to change an answer, do so.
Oral Tests/Structured Interviews
Many of the suggestions above apply to oral tests (i.e., structured interviews) as well. In an oral test, you will sit with a panel and answer questions that are job-related. Some panelists may ask how you would handle situations that may arise on the job. The purpose of the test is for you to demonstrate that you have the skills and abilities needed to solve job-related problems. Place yourself into the job, analyze what you should do, and consider the consequences of the actions you might take. A structured interview panel is interested in the content of your answers. Therefore, it is better to have a more thorough and complete answer.
Often, a complete answer requires you to bring up related points. Tell the panel what assumptions you are making and the reasons for your actions. Tell them what alternatives you consider and reject for good reasons. Do not skip the part of the answer you think the panelists already know. They have to hear you say it if they are going to give you credit for it. To prepare for an oral exam, practice talking about solving job-related problems. It may be helpful to investigate the kind of problems that occur in that type of job, so that you can become comfortable talking about them.