Breaking down the final score

[nylawyer]

I have already sent an e-mail to see how we can learn how our final score breaks down, but I am not expecting a reply.

If anyone has any ideas or insights, I'd love to hear them. Hard to decide whether to bother appealing, or even whether I should retake the test when offered if I don't know where my problems lie.

[arlene25]

I'd like to know too. I spent two grand on the interview and SA, which is a lot of money to me.

[justalawyer]

I'm with you ...

If you get any information, will you share it with us? thanks ...

[justalawyer]

Oh, and I traveled from the West Coast ... had to take a total of 4 days (2 days for each session ... my appointments were in the morning so I needed to travel the day before) off work for this insane process - plus the cost of 2 RT plane tickets, hotel, transporation, meals. I didn't total it up, but I'm estimating close to $3000 ... that IS a substantial sum for us government workers! What a racket. No, I'm not really bitter ... just wish I'd had more info going into the process so I could have made an informed decision.

[lawguardian]

nylawyer when you say you don't know whether you should take the "test" again are you just referring to reapplying when the position is reopened or something else?

[nylawyer]

what I am referring to is the rumored likelihood that OPM will start accepting applications again in about a year, at which point, if I am reading the e-mail correctly, I can ask to take the examination again. Before I woud do something like that, I'd like to see where I came up short- if my AR score is hopelessly low, then there is no point in wasting the time and money on this process again.

[hooligan]

When I received my score in 1998, the letter included a breakdown of how many points I earned in each of the catagories. I can not imagine trying to file an appeal without that information. My suggestion is that before you consider an appeal, try a freedom of information request asking about your score, the manner in which it was determined and your relative placement on the registry. You won't get a quick answer, but there will not be another round of hiring for perhaps another year anyway.

[tootsie]

Yeah- I think I'dappeal, but you have to give a basis. How does this sound: "I thought I did better on the examination" Winnable?

[nylawyer]

Tootsie- that's really the point of why I'd need to see the breakdown. I recognize that in all probability an appeal would be pointless. But, if I got the breakdown and saw that I got a zero on the SI, well, I'd appeal because I don't think anyone could possibly have done particularly well or badly on the ridiculous interview, and I'd have to assume that one of the interviewers mistook me for someone who beat him up in grade school.

[texasatty]

When I began thinking about appealing my terrible score and read that you had to give a basis for appealing, it struck me, "what a thing of beauty". Only the government could formulate a process where they don't give you a breakdown for the 3 components of your score, yet require a substantive basis for appeal.

Unless they give some indication of the breakdown, I've got no "basis" to appeal, other than, I thought I was better than this.

[shocked]

Just a theory. OPM/IFPTE prefer attorneys experienced outside SSA (what translates to a better product for the Agency is another issue - each view has some valid points).

The old register was reaching SSA attorneys, so it was easier to just scrap it. The new program now uses a 1-100 scale, based on:
The AR? It was a few hundred words on different areas, no real guidance and a lot of wiggle room (i.e., AR related to the practice of law or even what AR would score high?). How could the AR score explain an apparent disparity of 30-40 points between some obviously well qualified candidates (as they reported and I believe them) who almost certainly did reasonably well on the WD/SI.

But what about the long summary of your legal experience, broken down between litigation and administrative experience (plus your resume). That was not considered? Right.

Besides the ability to now weed out completely those without a lot of litigation background, OPM even got a bonus: they effectively eliminate the value of the veteran's preference.

If you can score the litigators 30+ points over anyone else, the veteran's preference of 10 points is worthless. I think that is a problem for OPM and I urge all the veterans to contact their respective congressional representatives to point out OPM's apparent end-run around the law.

[texasatty]

I'm not sure, "Shocked". This grading formula has effectively weeded me out and I've got 20 years of experience in federal district court and eight years experience in federal circuit court.

I get the feeling that OPM's grading system is a reaction to the old law suit, and their grading procedure and formula is a so rigid it doesn't permit the "graders" any discretion to recognize or credit various matters not on their "key word" list.

Did this process strike anyone else as being overly Orwellian, structured so strictly that form may overwhelm substance? I truly hope I'm not just being bitter.

Good luck to all.

[learnedhand]

This also raises questions when you consider that each area, AR, WD and SI were weighted equally. Added to that is the info that the weighting in the AR also put you on the register by allowing you to survive the cut.

So, with a max of 33.33 for each area, what does that break down to? I mean did you have to have an AR of 25 or so, for example, to make the cut? And what about those of us receiving a low score who think we did quite well, for example, on the WD and at least average on the SI or vice versa? I mean a 25 in each area should add up to 75, which means that if you got as high as 30 on the WD or as low as 20 on the SI, or the reverse, you'd still end up with a 75, not a 60, for example. When those of us who made the cut and think we did quite well on one area and OK on the other start adding points up, the outcome doesn't add up when compared to the score we received.

And why shouldn't the scoring be transparent? Then people know whether to apply or not and those who don't have valued skills can avoid wasting everyone's time and their own money. I think this may be an attempt to avoid suits, but the person who suggested rigid scoring criteria may be on the right track here.

[Pixie]

Learned: There is another theory going around, based on OPM's wording in the explanation, that each of the competencies--six in the AR and two each in the WD and SI were graded equally. Under that theory, the AR with six competencies would count as 75% of the final rating, and the WD and SI would each count 12%. It is hard to tell what OPM means from the wording. I guess eventually everyone will eventually find out. Pix.

[amporci]

My theory was that elements in the SI and elements in the WD were designed to evaluate the same areas ("competencies") specifically identified in the AR. I do not think we have been told (and wonder if we will ever be told) what percentage of each competency subscore came from the AR, WD and SI. I would love to know, however, because if the problem was my AR, I cannot do much about the underlying elements of my resume and experience. I could possibly improve on the WD/SI, if I decided to put myself through this again!

[shocked]

The other big question that (8 scores) raises would be is the value of 1 of the 6 (fairly brief) AR responses valued the same as the WD or the SI? I guess I fall back on my theory, as it is the logical answer given a lot of the history relative to veterans points and the value of ssa experience. But I readily admit it is almost silly to think OPM would go to such lengths. Time to move on and see what happens.

[nylawyer]

You will all be shocked to learn that I have not heard anything back from OPM since the original automated e-mail thanking me for my question. Guess if I want to appeal, it has to be done on faith.

[wdcsmp]

I took the WD on the 1st day it was offered (Mon. 7/23), and I was one of the 1st two candidates interviewed the next day, the first day of SIs. The SI process was horribly organized, the panels weren't ready at the scheduled time (we got started over over two hours late), and the 3-member panel was terrible, repeating questions asked by the prior questioner, no eye contact, etc. Overall, it was as miserable an interview experience as I've had, which doesn't appear to be uncommon, based on what I've seen here. I simply wanted out after awhile, and my guess is that it showed to some extent.

I just took a look at all application materials from the very start, trying to decide how or whether to appeal a relatively low score (62.65, no vet). I've gone over the competencies supposedly measured in the AR (6), WD (4), and SI (7). The initial application provides a defiinition for competencies in the AR (decisionmaking, interpersonal skills, oral communication, writing, judicial analysis, judicial management), and three competencies in the WD and SI are undefined (reasoning, self-management, stress tolerance). I can imagine trying to challenge a low rating on an panel member's assessment of my "reasoning" or "stress tolerance" as reflected during the SI.

Competencies aside, the very first thing I realized is that my application shows only 8.5 years of qualifying litigation experience, which makes me an utter neophyte in comparison with others here (I did transaction work for a number of years). My guess is that being relatively light on experience (AR) and having 3 interviewers who were 2.5 hours late in getting started on the first day of interviews (and seemed as though they could have used more fiber in their diets) (SI) probably kept me low, notwithstanding OPM's later explanation that the final score reflects an equal weighting of "the competencies," none of which include years of qualifying experience.

I'm leaning more towards a FOIA route to prep for when the list reopens in a year or so, but there are still three weeks to think it over. If I do appeal, it'll have to be on a fairly contrived basis, since we have no idea how they evaluated us on 17 different occasions and what role experience played, if any. BTW, nylawyer, I sent an email to aljapplication email address last night asking for a score breakdown and expect no response. It took them three weeks to respond to a very benign email I sent at the end of September asking when results would be sent out.

Thanks to all of you for your posts. It's been a very educational process for me, a first timer in this arena.

[nylawyer]

wdcsmp- thanks for your thoughts. I was thinking of appealing for the hell of it. I also am light on experience, which is why I'd really like to see the breakdown of my score- if it's that just made the initial cut after the application process and really stood no chance at all, then I know not to waste my time try to re-apply when the list is re-opened, since that's not going anywhere. If I bombed on one of the other sections, well, then I might want to give it another shot.

Please keep us informed if you get a response to your FOIA request.

[wdcsmp]

Thanks. I've not filed a FOIA request yet, but I'll likely do one if I don't appeal. I'll let you know if I hear anything from OPM in response to my email (unlikely).