Message from OPM about grading

[Deleted]

1

[owl]

Mar 20, 2014 12:54:53 GMT -5 sealaw90 said:

Mar 20, 2014 12:45:19 GMT -5 @wannabee2012 said:

Gary, you are repeating the mistake of others in adjusting the components to equal 100. That was not what was done. The components were added, then the total raw score was converted as I explained earlier to a 1-100 scale. It is a simple process, no mystery math or complicated formulas.

Wannabee, this is my interpretation of what you're saying;

This is what OPM means by rating each applicant with the other applicants.  The absolute top score on the test, whatever it was, is the one person who sets the denomniator for the rest of our scores to be divided by.  For example, one of us (NOT me) got a total score from all testing components and all competencies of 1,300.00.  Their score is divided by itself to get 1, move the decimal over two places to the right to get 100.  The rest of our measily scores are then divided by 1,300, move the decimal over two places to the right and there's your score.
Under this scenario, you will never be able to find out your "raw" score unless you know the top score for all applicants.

Correct??

I don't think OPM would make all the scoring for the register dependent on one person's top raw score, because there are always 10-point vets dropping in to test and if one of them exceeded the theretofore-established top score then either everyone's scores would have to be recalibrated or the new top score would exceed 100.00 (actually 110.00 after his/her 10 points are added).

I agree with wannabee2012. There were 13 competencies, and each was worth the same number of raw points. Pick any number you like as the number of raw points available per competency (10, 20, 35, whatever - only OPM knows). Multiply that number by 13, and you have the total number of points one could have scored on the whole exam (130, 260, 455, etc.). Your numerical rating is simply your percentage of your total competency score vs. the maximum possible score. Then 5 or 10 vet points are added if applicable.

The WD & SI (and for that matter the LBMT and SJT and maybe even the WS and EA) measured multiple competencies each. Applicants who didn't get to complete various parts of the exam will obviously have low scores across multiple competencies, and therefore end up with a low overall score. These, I believe, are the people whose scores are in the 1-to-40s but obviously those scores are known only to OPM (and do not show up on the polls here) because you only get a score if you make the register.

The hypothetical 1.00 scorer probably does not exist but there most certainly is a real person who has the lowest (or tied for lowest) score on the register. Maybe that person was right on the cut line for moving on to phase 3 and then got the minimum scores on both the WD & SI (and had a low score on the LBMT too). But even that person would at least have had all the various opportunities to rack up competency points by taking all the parts of the exam, so that person's score has to be well above 1.00 and is probably around 50 since there are low-50s scores reported here and I doubt the polling here is wildly unrepresentative.

[Deleted]

1

[anotherfed]

This is why our final numerical rating is a rank on a scale of 1-100. I don't believe the "1" rating is hypothetical. If the OPM email says that everyone who has a rating between 1 and 100, then a rating of 1 gets on the Register. I don't think the final numerical rating is a permutation of our aggregate raw scores on the various components, other than to put us on a list in descending order. Anyone who did not get the minimum score on the SI or the WD was not put on the list. All others who completed the testing were.

[gunner]

Mar 20, 2014 12:54:53 GMT -5 sealaw90 said:

Mar 20, 2014 12:45:19 GMT -5 @wannabee2012 said:

Gary, you are repeating the mistake of others in adjusting the components to equal 100. That was not what was done. The components were added, then the total raw score was converted as I explained earlier to a 1-100 scale. It is a simple process, no mystery math or complicated formulas.

Wannabee, this is my interpretation of what you're saying;

This is what OPM means by rating each applicant with the other applicants.  The absolute top score on the test, whatever it was, is the one person who sets the denomniator for the rest of our scores to be divided by.  For example, one of us (NOT me) got a total score from all testing components and all competencies of 1,300.00.  Their score is divided by itself to get 1, move the decimal over two places to the right to get 100.  The rest of our measily scores are then divided by 1,300, move the decimal over two places to the right and there's your score.
Under this scenario, you will never be able to find out your "raw" score unless you know the top score for all applicants.

Correct??

I think you have correctly described Wannabee's post, but I don't think it's right because it would establish only the applicant's score as a percentage of the highest scorer's score. It would not be a 1-100 scale as described in the scoring email, which says:

"The total competency scores of all applicants were put on a 1 to 100 scale to establish each applicant's numerical rating, excluding veterans' preference. In this scale, the lowest possible score is 1 and the highest possible score is 100. A score of 1 would mean only that this individual had the lowest rating relative to other candidates on the register (or was tied for the lowest rating)"

If Wannabee were right, then, assuming a 1300 was the highest raw score, the high scorer does get a 1.00 or 100 once you move the decimal points. But it would take a 13 raw score to get a 1.00, and that would not be the lowest score because if you scored 12 you would get a 0.92.

You could get closer with this formula:

Call Highest Raw Score = h
Lowest Raw Score = L
Applicant Raw Score = a
NOR= n

      (a - L)
n =  --------- x 100
      (h - L)

That would give you the applicant's raw score percentage

So if the highest raw score was 1300 and the lowest was 1000, the 1300 person's core NOR is (1300 - 1000)/(1300 - 1000) * 100; or 1.00 * 100.

The problem is that the lowest scorer's NOR would be undefined because his numerator would be zero (his score minus the lowest score) and you cannot divide by zero. We can't just give that person a 1 and go from there, because someone who scored one raw point higher (let's say 1001) would wind up with a NOR less than 1. (1/300 = 0.0033, or 0.33 after multiplying by 100).

I think the solution would be:

       (a - L) + 1                      1
n = ( ------------- x 100 ) + 1 - (------------ )
       (h - L) + 1                  (h - L) + 1

What that does is take care of the undefined lowest scorer problem by adding one to the numerator & denominator in the first part of the function. Our lowest scorers in the above scenario (1000-1300 raw score range) would get 1/301 & 2/301, or 0.33 and 0.67 after multiplying by 100. Then you add 1 to make sure they're above 1, and subtract out the lowest scorer's percentage to make sure he gets a NOR of exactly 1. So the lowest scorer's NOR is 1.00. The second lowest scorer is 1.34, etc.

That works all the way up to the top couple of scorers, who will have NORs of 100.66 and 100.33. I'm sure there's a similar fix for those but my head's hurting now. I didn't become a lawyer for the math.

[sealaw90]

Mar 20, 2014 14:10:23 GMT -5 @wannabee2012 said:

Sealaw. I stand corrected. It is simpler than I made it. Of course you don't use the high scorer. You use max possible score. The math is the same for a simple conversion to 1-100 scale.

Yes, I agree with you.  It was stream of conciousness drafting of my previous post in order to wrap my head around this subject.

Quite honestly, does it really matter?  How could this possibly be used in an appeal?  I hope more people spend lots of time thinking about this instead of taking their references out to lunch.  Heck, everyone else on this board should be thinking about OPM scoring rubrics, it might increase my chances of getting hired off the cert that I (eventually) get on!!

[gunner]

Mar 20, 2014 14:10:23 GMT -5 @wannabee2012 said:

Sealaw. I stand corrected. It is simpler than I made it. Of course you don't use the high scorer. You use max possible score. The math is the same for a simple conversion to 1-100 scale.

That would be a sensible way to do it, but it would only show the percentage you got of the total. It would not give you something where "the lowest possible score is 1."

But perhaps that's the way to fix the problem in my formula above. Maybe instead of defining h as the highest scorer, it's the highest score possible. After all, the instructions do not say that 100 is the highest scoring applicant, just that it's the highest score possible. So the highest scorer may be a NOR of 89. That would also make sense to allow for 10 point preferences with the theoretically possible but practically unlikely scenario of someone with a >100 NOR

[JudgeRatty]

Mar 20, 2014 14:45:05 GMT -5 gunner said:

Mar 20, 2014 12:54:53 GMT -5 sealaw90 said:

Wannabee, this is my interpretation of what you're saying;

This is what OPM means by rating each applicant with the other applicants.  The absolute top score on the test, whatever it was, is the one person who sets the denomniator for the rest of our scores to be divided by.  For example, one of us (NOT me) got a total score from all testing components and all competencies of 1,300.00.  Their score is divided by itself to get 1, move the decimal over two places to the right to get 100.  The rest of our measily scores are then divided by 1,300, move the decimal over two places to the right and there's your score.
Under this scenario, you will never be able to find out your "raw" score unless you know the top score for all applicants.

Correct??

I think you have correctly described Wannabee's post, but I don't think it's right because it would establish only the applicant's score as a percentage of the highest scorer's score. It would not be a 1-100 scale as described in the scoring email, which says:

"The total competency scores of all applicants were put on a 1 to 100 scale to establish each applicant's numerical rating, excluding veterans' preference. In this scale, the lowest possible score is 1 and the highest possible score is 100. A score of 1 would mean only that this individual had the lowest rating relative to other candidates on the register (or was tied for the lowest rating)"

If Wannabee were right, then, assuming a 1300 was the highest raw score, the high scorer does get a 1.00 or 100 once you move the decimal points. But it would take a 13 raw score to get a 1.00, and that would not be the lowest score because if you scored 12 you would get a 0.92.

You could get closer with this formula:

Call Highest Raw Score = h
Lowest Raw Score = L
Applicant Raw Score = a
NOR= n

      (a - L)
n =  --------- x 100
      (h - L)

That would give you the applicant's raw score percentage

So if the highest raw score was 1300 and the lowest was 1000, the 1300 person's core NOR is (1300 - 1000)/(1300 - 1000) * 100; or 1.00 * 100.

The problem is that the lowest scorer's NOR would be undefined because his numerator would be zero (his score minus the lowest score) and you cannot divide by zero. We can't just give that person a 1 and go from there, because someone who scored one raw point higher (let's say 1001) would wind up with a NOR less than 1. (1/300 = 0.0033, or 0.33 after multiplying by 100).

I think the solution would be:

       (a - L) + 1                      1
n = ( ------------- x 100 ) + 1 - (------------ )
       (h - L) + 1                  (h - L) + 1

What that does is take care of the undefined lowest scorer problem by adding one to the numerator & denominator in the first part of the function. Our lowest scorers in the above scenario (1000-1300 raw score range) would get 1/301 & 2/301, or 0.33 and 0.67 after multiplying by 100. Then you add 1 to make sure they're above 1, and subtract out the lowest scorer's percentage to make sure he gets a NOR of exactly 1. So the lowest scorer's NOR is 1.00. The second lowest scorer is 1.34, etc.

That works all the way up to the top couple of scorers, who will have NORs of 100.66 and 100.33. I'm sure there's a similar fix for those but my head's hurting now. I didn't become a lawyer for the math.

It was traumatic enough to have logic on the menu, but now algebra? I swore off statistics and analytical math problems when I finished my bachelor's degree. LOL!

[decadealj]

As I recall, the whole basis of the Adzell(?) litigation was adding VP to the final raw score so that many vets had scores over 100. Under the old (very old)1985 to 1994 or so process, the highest score was a 97 or something, and everyone knew how the scoring was done and where they fit on the register because all scores in the testing class were listed. Don't know what good or harm all the speculation does but its kind of meaningless when you consider what happens in the rest of the appointment process. Get some sleep- get a massage.

[JudgeRatty]

I will sure be glad when the first cert comes out so my head can stop hurting from all this MATH! LOL! Several people I talked to today told me that Friday was a "favorite" day to send out the cert emails. Favorite may not be appropriate, more like usual or past history. So keep the faith people! One step closer for many of the people here so good luck to you all!

[JudgeRatty]

Mar 20, 2014 16:51:00 GMT -5 @devildog said:

Any other confirmed or unconfirmed BS that they have issued a cert?

I did have one person who knows tell me they are actively working the transfer list as of yesterday. I guess they don't necessarily have to have that done before starting the references/interviewing process with a new cert.

My HOCALj mentioned no emails have gone out for assistance with interviews yet but also said that they usually (at least have in the past) sent the cert notifications on Fridays. So this is a non answer but I figure once the request for interviewers is out that would be meaningful.

[funkyodar]

Mar 20, 2014 14:03:09 GMT -5 owl said:

Mar 20, 2014 12:54:53 GMT -5 sealaw90 said:

Wannabee, this is my interpretation of what you're saying;

This is what OPM means by rating each applicant with the other applicants.  The absolute top score on the test, whatever it was, is the one person who sets the denomniator for the rest of our scores to be divided by.  For example, one of us (NOT me) got a total score from all testing components and all competencies of 1,300.00.  Their score is divided by itself to get 1, move the decimal over two places to the right to get 100.  The rest of our measily scores are then divided by 1,300, move the decimal over two places to the right and there's your score.
Under this scenario, you will never be able to find out your "raw" score unless you know the top score for all applicants.

Correct??

I don't think OPM would make all the scoring for the register dependent on one person's top raw score, because there are always 10-point vets dropping in to test and if one of them exceeded the theretofore-established top score then either everyone's scores would have to be recalibrated or the new top score would exceed 100.00 (actually 110.00 after his/her 10 points are added).

I agree with wannabee2012. There were 13 competencies, and each was worth the same number of raw points. Pick any number you like as the number of raw points available per competency (10, 20, 35, whatever - only OPM knows). Multiply that number by 13, and you have the total number of points one could have scored on the whole exam (130, 260, 455, etc.). Your numerical rating is simply your percentage of your total competency score vs. the maximum possible score. Then 5 or 10 vet points are added if applicable.

The WD & SI (and for that matter the LBMT and SJT and maybe even the WS and EA) measured multiple competencies each. Applicants who didn't get to complete various parts of the exam will obviously have low scores across multiple competencies, and therefore end up with a low overall score. These, I believe, are the people whose scores are in the 1-to-40s but obviously those scores are known only to OPM (and do not show up on the polls here) because you only get a score if you make the register.

The hypothetical 1.00 scorer probably does not exist but there most certainly is a real person who has the lowest (or tied for lowest) score on the register. Maybe that person was right on the cut line for moving on to phase 3 and then got the minimum scores on both the WD & SI (and had a low score on the LBMT too). But even that person would at least have had all the various opportunities to rack up competency points by taking all the parts of the exam, so that person's score has to be well above 1.00 and is probably around 50 since there are low-50s scores reported here and I doubt the polling here is wildly unrepresentative.

This, owl's theory, makes the most sense to me. It accounts for the curve and whatnot.

Most importantly it answers the "where are the low scores" question. Those folks are there, its just that their overall score in the competencies is low because they failed to meet the minimum on the wd and si. But for that minimum (and based on our sd poll) this register would be about 25% larger and it would have scores in the 50s, 40s and 30s typical of past registers.

Prior to this testing, an applicant submitted their "accomplishment record" and it was graded and assigned a point value. Those in the higher scored subgroup were invited to DC. How was the cut line determined? Easy, you just decide how big a register you wanted and went down the AR rankings till you had enough. Then, all dc test takers were scored and, because there was no minimum thresholds, everyone made the register. Some did so with scores in the 30s but still made the register.

Now, the higher scored subgroup was pulled from the scores on the online component. I assume the cut line was still based on some number they had in mind for dc.

But, unlike last time, there is now the wd and si minimums. Meeting those minimums, combined with your online component scores, essentially meant your overall score was gonna be in the high 50s or low 60s. Missing the minimum meant scores in the lower 50s, 40s and 30s. We aren't seeing those scores on this reg because they were cut.

So a 60 on this reg is on the low end of the reg, but its really no different than a 60 on the old register. Its just that in the past looked better in comparison to the reported lower scores. Now, instead, opm can tell odar "we did a more targeted test and this new register is the equivalent to the top of the old registers" because they essentially cut the bottom quarter that would, in the past, have been included.

[termoyle]

I cannot add anything useful about the 1-100 scale.  But the "random numbers" (as Burghbum noted earlier) are likely imbedded codes.  They did not translate into the special characters they represent, but this appears to be the key:

    1920 = ' (apostrophe -  as in veterans')
    2220 = tab (to indent the paragraph)
    1C20 = " (as in "failed")

There.  I have waited a year to have something useful to say.  Now someone will probably point out that Funky had already posted about this before and everybody already knew it.

[funkyodar]

Mar 20, 2014 17:20:09 GMT -5 termoyle said:

I cannot add anything useful about the 1-100 scale.  But the "random numbers" (as Burghbum noted earlier) are likely imbedded codes.  They did not translate into the special characters they represent, but this appears to be the key:

    1920 = ' (apostrophe -  as in veterans')
    2220 = tab (to indent the paragraph)
    1C20 = " (as in "failed")

There.  I have waited a year to have something useful to say.  Now someone will probably point out that Funky had already posted about this before and everybody already knew it.

Damn. Foiled again. Welcome to the board.

[71stretch]

Mar 20, 2014 14:13:52 GMT -5 anotherfed said:

This is why our final numerical rating is a rank on a scale of 1-100. I don't believe the "1" rating is hypothetical. If the OPM email says that everyone who has a rating between 1 and 100, then a rating of 1 gets on the Register. I don't think the final numerical rating is a permutation of our aggregate raw scores on the various components, other than to put us on a list in descending order. Anyone who did not get the minimum score on the SI or the WD was not put on the list. All others who completed the testing were.

The final rating of 1 doesn't get in the register if they don't get the minimum on the SI or WD. I don't know how anyone with a total score of 1 got out of stage 1.

[robespierre]

Gunner's post put this issue officially beyond my realm of understanding. I'm swearing off this issue until a cert comes out and a poll indicates what scores got onto it. LOL. Signing off this thread.

[funkyodar]

Here's some numbers to consider:

At last report, something near 6000 people submitted an application by 3/15/13. Some were cut because their apps weren't "complete." More were told they didn't have the 7 years and were cut. A bunch didn't make the "higher scored subgroup" after the online component. Almost 25% (if our polling is accurate) had the honor of going to dc only to then be cut.

If our speculation is close, of that 6000, around 800 made the register. Thats 13.33%. Over 86% of people who applied didn't makethe register for whatever reason.

When you stated this, as one of the 6000 and with the fact we now know that odar will hire 90 this year, you had only a 1.5% chance of being an alj by the end of fiscal 2014.

now that you are on the register? You have about a 35 to 40% chance of making the cert (based on raw numbers, chances are greater or lesser based on individual scores and gals of course). Once on a cert, you have close to a 33% chance of getting a gig (not a perfect third because the cert will be more than exactly 3x the slots, but who knows how much more).

Many of us have significant disappointment ahead. Maybe more of a disapointment than was experienced by those cut at earlier stages. But just by getting on the reg, whatever your score, you still have a shot. And its gone from lottery chances to 3 card monty.

[Ace Midnight]

Mar 21, 2014 10:44:06 GMT -5 funkyodar said:

the fact we now know that odar will hire 90 this year.

That remains the intention, and a strong suggestion of that many in 2015. I heard it, face-to-face from the executive at ODAR who would know.

[sealaw90]

Mar 21, 2014 10:44:06 GMT -5 funkyodar said:

Here's some numbers to consider:

At last report, something near 6000 people submitted an application by 3/15/13. Some were cut because their apps weren't "complete." More were told they didn't have the 7 years and were cut. A bunch didn't make the "higher scored subgroup" after the online component. Almost 25% (if our polling is accurate) had the honor of going to dc only to then be cut.

If our speculation is close, of that 6000, around 800 made the register. Thats 13.33%. Over 86% of people who applied didn't make the register for whatever reason.

When you stated this, as one of the 6000 and with the fact we now know that odar will hire 90 this year, you had only a 1.5% chance of being an alj by the end of fiscal 2014.

funky, the other number that the 6,000 apps remind me of - the number of them who are a "1" on the NOR.  That would be the 2,000 or so who didn't make it past the initial application - I'm feelin' real smart with my giant bell curve of a score

[privateatty]

Mar 21, 2014 12:09:08 GMT -5 sealaw90 said:

Mar 21, 2014 10:44:06 GMT -5 funkyodar said:

Here's some numbers to consider:

At last report, something near 6000 people submitted an application by 3/15/13. Some were cut because their apps weren't "complete." More were told they didn't have the 7 years and were cut. A bunch didn't make the "higher scored subgroup" after the online component. Almost 25% (if our polling is accurate) had the honor of going to dc only to then be cut.

If our speculation is close, of that 6000, around 800 made the register. Thats 13.33%. Over 86% of people who applied didn't make the register for whatever reason.

When you stated this, as one of the 6000 and with the fact we now know that odar will hire 90 this year, you had only a 1.5% chance of being an alj by the end of fiscal 2014.

funky, the other number that the 6,000 apps remind me of - the number of them who are a "1" on the NOR.  That would be the 2,000 or so who didn't make it past the initial application - I'm feelin' real smart with my giant bell curve of a score

As well you should. Now do the final lap and keep your cool and your sense of humor.