|
Post by zebra51 on May 18, 2015 13:21:48 GMT -5
Even more to the contrary those that were cut after the wd and si were cut for failing to make a "minimum required score" on those elements. We were all told of such requirements. The scores were not, I presume, arbitrary. The only thing that seems fair in that regard is to have appeal rights. I have to disagree. I do presume that the WD and SI scores were arbitrary. While the announcement said "minimum required score" , go back and read the thread from those that did not make the WD ( aljdiscussion.proboards.com/thread/2469/thoughts-appealing-denial-written-demonstration?page=2 ). I believe the "minimum required score" was most likely set arbitrarily to set a quota of the number of people getting a NOR.
|
|
|
Post by funkyodar on May 18, 2015 13:44:39 GMT -5
From the announcement:
"Section 1: WD - The purpose of the WD is to evaluate an applicant's ability to prepare a clear, concise, and well-reasoned legal decision of the type that one might be expected to write if employed as an ALJ. The WD is scheduled for 5 hours to allow time for instructions and other administrative processes, but actual testing time is 4 hours. The WD is conducted in a proctored environment using a laptop computer provided by OPM, and will be administered in one location in the Washington, DC area.
If you do not receive the required minimum score on the WD, you will not receive a final numerical rating and will not be placed on the ALJ register.
SI - The objective of the SI is to evaluate an applicant's responses to competency-based questions related to being an ALJ. A panel will conduct the interview and evaluate the responses provided by the applicant. The interview will last approximately one hour, but you will need to arrive early to allow time for instructions. The SI will be administered in one location in the Washington, DC area.
If you do not receive the required minimum score on the SI, you will not receive a final numerical rating and will not be placed on the ALJ register."
Versus:
"If the score for your performance on the SJT, Writing Sample, and Experience Assessment is within the range for the higher-scored sub-group of all the eligible applicants, you will be invited to participate in the Written Demonstration and Logic-Based Measurement Test, and the Structured Interview. You will be notified via email regarding when and where to report for the Written Demonstration and Logic-Based Measurement Test, and for the Structured Interview.
If the score for your performance on the SJT, Writing Sample, and Experience Assessment is not within the range for the higher-scored sub-group of eligible applicants, you will be notified via email that you will no longer be considered for this current ALJ Job Opportunity Announcement. You will not be invited to participate in the Written Demonstration and Logic-Based Measurement Test or the Structured Interview. Your official Notice of Results and a notice describing your appeal rights will be issued after the U.S. Office of Personnel Management completes the administration of the ALJ examination for the entire group that participated in the examination."
I get what you are saying Zebra, but it seems pretty clear to me that the online component was designed to rank applicants into subgroups based on ranges of scores. An arbitrary ranking designed solely for management of the applicant pool brought to DC. Whereas, it was fundamentally necessary for each applicant to meet the set minimum score on the SI and WD. Without doing so, you could not get on the register. That, to me, says they set that score based on something.
|
|
|
Post by christina on May 18, 2015 13:49:31 GMT -5
i am not sure i agree as opm could drop the minimum score from the WD and SI down too. i only glanced at the language you posted funky but a quick glance does not prohibit opm from revising the minimum score downward. and thanks for getting the actual language on here funky.
|
|
|
Post by gary on May 18, 2015 14:24:03 GMT -5
I agree with Funky. The purpose of the cutoff for the online components was to determine who should be invited to test in DC. It allowed OPM to control the numbers to be tested by inviting however many it wanted to test based on the online testing scores. The cutoff scores for the WD and SI were to keep off the register candidates OPM adjudged would not be good ALJs (for one reason or another). OPM could be wrong about that for several reasons, but that at least was the purpose for those cutoffs. It is one thing for OPM to change the cutoff for testing in DC where candidates can prove themselves, and quite another to put on the register candidates who do not make the minimum grade OPM believes is necessary for an ALJ.
|
|
|
Post by alrifle on May 18, 2015 14:31:22 GMT -5
From the announcement: "Section 1: WD - The purpose of the WD is to evaluate an applicant's ability to prepare a clear, concise, and well-reasoned legal decision of the type that one might be expected to write if employed as an ALJ. The WD is scheduled for 5 hours to allow time for instructions and other administrative processes, but actual testing time is 4 hours. The WD is conducted in a proctored environment using a laptop computer provided by OPM, and will be administered in one location in the Washington, DC area. If you do not receive the required minimum score on the WD, you will not receive a final numerical rating and will not be placed on the ALJ register. SI - The objective of the SI is to evaluate an applicant's responses to competency-based questions related to being an ALJ. A panel will conduct the interview and evaluate the responses provided by the applicant. The interview will last approximately one hour, but you will need to arrive early to allow time for instructions. The SI will be administered in one location in the Washington, DC area. If you do not receive the required minimum score on the SI, you will not receive a final numerical rating and will not be placed on the ALJ register." Versus: "If the score for your performance on the SJT, Writing Sample, and Experience Assessment is within the range for the higher-scored sub-group of all the eligible applicants, you will be invited to participate in the Written Demonstration and Logic-Based Measurement Test, and the Structured Interview. You will be notified via email regarding when and where to report for the Written Demonstration and Logic-Based Measurement Test, and for the Structured Interview. If the score for your performance on the SJT, Writing Sample, and Experience Assessment is not within the range for the higher-scored sub-group of eligible applicants, you will be notified via email that you will no longer be considered for this current ALJ Job Opportunity Announcement. You will not be invited to participate in the Written Demonstration and Logic-Based Measurement Test or the Structured Interview. Your official Notice of Results and a notice describing your appeal rights will be issued after the U.S. Office of Personnel Management completes the administration of the ALJ examination for the entire group that participated in the examination." I get what you are saying Zebra, but it seems pretty clear to me that the online component was designed to rank applicants into subgroups based on ranges of scores. An arbitrary ranking designed solely for management of the applicant pool brought to DC. Whereas, it was fundamentally necessary for each applicant to meet the set minimum score on the SI and WD. Without doing so, you could not get on the register. That, to me, says they set that score based on something. Thanks for posting these requirements, funky. I think it is clear that OPM approached the two rounds differently. However, it is still possible that, while not per se arbitrary, the minimum it set at the WD/SI round was created to allow a certain number of the (roughly 1100?) DC testers onto the register, as I think zebra may be suggesting. I could be wrong but I don't believe we know if the minimum was set before the DC phase or after.
|
|
|
Post by alrifle on May 18, 2015 14:43:29 GMT -5
I agree with Funky. The purpose of the cutoff for the online components was to determine who should be invited to test in DC. It allowed OPM to control the numbers to be tested by inviting however many it wanted to test based on the online testing scores. The cutoff scores for the WD and SI were to keep off the register candidates OPM adjudged would not be good ALJs (for one reason or another). OPM could be wrong about that for several reasons, but that at least was the purpose for those cutoffs. It is one thing for OPM to change the cutoff for testing in DC where candidates can prove themselves, and quite another to put on the register candidates who do not make the minimum grade OPM believes is necessary for an ALJ. I think you are right. However, this entire thread is a discussion addressing a significant alteration in what is explicitly stated in the job announcement. If OPM can alter its stated process in one respect, it can alter it in another respect just as easily. I am not saying that OPM will do this - I just don't think it's outside of the realm of possibilities.
|
|
|
Post by moopigsdad on May 18, 2015 14:49:06 GMT -5
Christina, Thanks. That's why I prefaced my comments with maybe this isn't appropriate - I'm not surprised to hear that but since I didn't know for sure, thought I'd inquire. Shows how much of a neophyte I am. Ironic, since I've had my eye on an ALJ position as an option for more than 30 years - but perhaps not from a very informed point of view. I supposed I won't be hearing back until after June 2 but I will be on pins and needles since then. I sure hope I didn't hit the wrong button but since it was beside "No, I am not interested" I am pretty sure I was careful enough to give the right answer! Now to work toward passage of the other levels and hopefully get on the register!! Good luck, everyone!! H2BALJ please make sure your email address on USAJobs is correct. Secondly, please carefully check your regular email along with any spam in your email account to make sure you do not miss the email from OPM this time. It is imperative you constantly check daily for an email (both regular and spam) for any testing links for Part 2 of the Examination. I hope this helps you get past this stage and to Part 3 in D.C. Once again, please always check your email prior to deleting anything in it, spam or otherwise, from your email account. Good luck!
|
|
|
Post by keepsake on May 18, 2015 15:01:38 GMT -5
If I understand Hope2BALJ's situation, he was deemed qualified for Round 1 but either missed the notification to move on with Round 2 or did not get the email and appealed. And yet, OPM wouldn't necessarily have had to rule on that appeal to pass Hope2BALJ on to Round 2 testing as it might have been deemed a lack of participation/interest to proceed so they have asked whether these type of people are still interested and opened it back up to those preliminary qualified (i.e., passed Round 1). And I note that OPM's email to Hope2BALJ says nothing about granting an appeal.
I am not sure how many people are in this situation, but I for one would be very interested in hearing from someone, anyone, who has gotten a substantive response to their appeal at whatever stage. Maybe Hope2BALJ is in a very small subset, but it would be interesting to know what other "class" of people have gotten the nod to proceed from OPM and I look forward to hearing details from others getting this kind of green light.
Appeal determinations might be forthcoming during the next few months but I am not necessarily hopeful for a quick response with respect to such appeals as OPM seems to be adjusting the qaulifying scores to get more people to DC and not necessarily opening the process to what will be successful appellants at the immediate moment.
|
|
|
Post by Gaidin on May 18, 2015 15:02:53 GMT -5
I agree with Funky. The purpose of the cutoff for the online components was to determine who should be invited to test in DC. It allowed OPM to control the numbers to be tested by inviting however many it wanted to test based on the online testing scores. The cutoff scores for the WD and SI were to keep off the register candidates OPM adjudged would not be good ALJs (for one reason or another). OPM could be wrong about that for several reasons, but that at least was the purpose for those cutoffs. It is one thing for OPM to change the cutoff for testing in DC where candidates can prove themselves, and quite another to put on the register candidates who do not make the minimum grade OPM believes is necessary for an ALJ. I think you are right. However, this entire thread is a discussion addressing a significant alteration in what is explicitly stated in the job announcement. If OPM can alter its stated process in one respect, it can alter it in another respect just as easily. I am not saying that OPM will do this - I just don't think it's outside of the realm of possibilities. I disagree. I am not an expert in testing but I have had to review testing protocols from a legal perspective before. The way I read the text posted by funkyodar above is that the SJT/WS grade is given and they will place people in subgroups based upon percentages. The WD/SI component had predetermined minimum scores. Those minimum scores were not based upon how many candidates it produced but instead ensuring that OPM could certify to any agency that the candidates met not only experience qualifications but also had met a threshold for "judginess" (TM). That means they can easily continue to send lower scoring subgroups through to DC without breaking the rules laid out on the testing because its not a passing score. A passing score means you have sufficient judginess (TM). I think that the easy to grant appeals like Hope2B's above will also start resolving themselves. If someone missed the email saying they got through to DC they will likely get a similar email inviting them on. I am a little concerned everybody keeps tossing the word arbitrary around. That is a really serious accusation to toss at an agency and is usually accompanied by the word capricious. I wouldn't toss it out there without some evidence to back it up. Lastly, please do not interpret this as me saying that anyone doesn't deserve to make the register. I know several people that I met along the trail who I believed had the experience and qualities necessary to be good ALJs. I believe many more people who got cut loose before the end of the journey were also quality candidates. I wish everybody seeking an appeal good luck and I hope you all get a legitimate opportunity for your appeal to be heard.
|
|
|
Post by gary on May 18, 2015 15:23:09 GMT -5
Well said, Gaidin. The point about people arbitrarily tossing about the word arbitrary especially needed to be made.
OPM is staffed by trained and conscientious professional people who know more about this whole process than any of us ever will. I may disagree with something they do. I certainly don't understand everything about this process. But I never make the mistake of underestimating the diligence and professionalism with which OPM's staff perform their duties.
|
|
|
Post by zebra51 on May 18, 2015 15:25:41 GMT -5
You are being sent this message because you meet ALL of the conditions below: Submitted an application for the Administrative Law Judge (ALJ) examination, Job Opportunity Announcement (JOA) ALJ2013-847661 during the filing period. Cleared the preliminary qualifications screening. Received an IMIN rating on the ALJ examination Notice of Results. Did not start any portion of the three Online Component assessments (i.e., Situational Judgment Test (SJT), Writing Sample (WS), and Experience Assessment (EA)). Did not re-apply and did not re-take the ALJ examination pursuant to 5 CFR 332.311. Looks like OPM may have preemptively addressed some of the issues from MSPB case Zane v OPM. www.mspb.gov/netsearch/viewdocs.aspx?docnumber=1064958&version=1069231&application=ACROBAT Further, the appellant appears to have identified a fatal flaw in the online ALJ examination process. In this instance, the appellant, despite her best efforts, could not even access the examination stage, even though she apparently qualified to do so. As a result, this glitch in the online application process prevented the appellant from demonstrating whether she met the basic requirements for the position as provided for under 5 C.F.R. § 300.103. By alleging that the online application process was flawed and as a result it violated one of the “basic requirements” for employment practices set forth in 5 C.F.R. § 300.103, the appellant has raised a clear question of whether OPM’s online ALJ examination process may constitute an employment practice over which the Board has jurisdiction. Thus, the appellant has made a nonfrivolous allegation that the employment practice violated a basic requirement of 5 C.F.R. § 300.103.
|
|
|
Post by gary on May 18, 2015 15:32:34 GMT -5
You are being sent this message because you meet ALL of the conditions below: Submitted an application for the Administrative Law Judge (ALJ) examination, Job Opportunity Announcement (JOA) ALJ2013-847661 during the filing period. Cleared the preliminary qualifications screening. Received an IMIN rating on the ALJ examination Notice of Results. Did not start any portion of the three Online Component assessments (i.e., Situational Judgment Test (SJT), Writing Sample (WS), and Experience Assessment (EA)). Did not re-apply and did not re-take the ALJ examination pursuant to 5 CFR 332.311. Looks like OPM may have preemptively addressed some of the issues from MSPB case Zane v OPM. www.mspb.gov/netsearch/viewdocs.aspx?docnumber=1064958&version=1069231&application=ACROBAT Further, the appellant appears to have identified a fatal flaw in the online ALJ examination process. In this instance, the appellant, despite her best efforts, could not even access the examination stage, even though she apparently qualified to do so. As a result, this glitch in the online application process prevented the appellant from demonstrating whether she met the basic requirements for the position as provided for under 5 C.F.R. § 300.103. By alleging that the online application process was flawed and as a result it violated one of the “basic requirements” for employment practices set forth in 5 C.F.R. § 300.103, the appellant has raised a clear question of whether OPM’s online ALJ examination process may constitute an employment practice over which the Board has jurisdiction. Thus, the appellant has made a nonfrivolous allegation that the employment practice violated a basic requirement of 5 C.F.R. § 300.103. For clarity's sake, the quote is from hope2balj.
|
|
|
Post by keepsake on May 18, 2015 15:43:28 GMT -5
I see the point here, but I don't think it is a surprise that people use the term "arbitrary" given the APA standards for judicial review. I mean what else do folks have to hang their hats on when challenging agency action (here OPM's issuance of what they believe to be erroneous NORs) than the phrase - "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law" or somethig along those lines.
As such, I don't think people are generally trying to call into question the integrity or diligence of OPM staff (or contractors for that matter if they were hired to do the assessments/grading) who reviewed/graded the applications/tests. But one is stuck with the rules of the game one is playing and if you appeal, many people probably felt they had better have done something other than just say - wow - I disagree with your NOR, pretty please look again . . . .
I could be wrong about this but I at least can appreciate how problematic it is/was to deal with this new test with what surely must be limited staffing issues and a large number appeals. Doesn't make having gotten axed out any easier or from becoming frustrated at the pace of resolution for appeals.
|
|
|
Post by JudgeRatty on May 18, 2015 15:57:59 GMT -5
To add to the point Gaidin made about the word "arbitrary"... just because WE do not know what the actual threshold number was that they used for the SJT/EA phase, or the actual numerical value to be able to make the register, does not mean these numbers are arbitrary. It merely means we are not privy to that information. An agency that is well versed in litigation on the hiring process no doubt has everything lined up and did so prior to the testing.
|
|
|
Post by gary on May 18, 2015 15:58:20 GMT -5
Each appeal pending before OPM's ALJ Appeals Panel is based on a claim that a rating was assigned in error. These appeals do not require an action to have been arbitrary.
|
|
|
Post by alrifle on May 18, 2015 16:02:36 GMT -5
I think you are right. However, this entire thread is a discussion addressing a significant alteration in what is explicitly stated in the job announcement. If OPM can alter its stated process in one respect, it can alter it in another respect just as easily. I am not saying that OPM will do this - I just don't think it's outside of the realm of possibilities. I disagree. I am not an expert in testing but I have had to review testing protocols from a legal perspective before. The way I read the text posted by funkyodar above is that the SJT/WS grade is given and they will place people in subgroups based upon percentages. The WD/SI component had predetermined minimum scores. Those minimum scores were not based upon how many candidates it produced but instead ensuring that OPM could certify to any agency that the candidates met not only experience qualifications but also had met a threshold for "judginess" (TM). That means they can easily continue to send lower scoring subgroups through to DC without breaking the rules laid out on the testing because its not a passing score. A passing score means you have sufficient judginess (TM). I think that the easy to grant appeals like Hope2B's above will also start resolving themselves. If someone missed the email saying they got through to DC they will likely get a similar email inviting them on. I am a little concerned everybody keeps tossing the word arbitrary around. That is a really serious accusation to toss at an agency and is usually accompanied by the word capricious. I wouldn't toss it out there without some evidence to back it up. Lastly, please do not interpret this as me saying that anyone doesn't deserve to make the register. I know several people that I met along the trail who I believed had the experience and qualities necessary to be good ALJs. I believe many more people who got cut loose before the end of the journey were also quality candidates. I wish everybody seeking an appeal good luck and I hope you all get a legitimate opportunity for your appeal to be heard. Okay, you may disagree, but I was merely pointing out that the announcement makes it sound pretty final as to the online component as it states that if you are not in the higher scored subgroup, "you will no longer be considered". That sounds like finality to me, and the only prescribed solution for those in that situation - from the announcement - is an appeal. Now we hear differently, and I think that's fine. I am not sure what you are disagreeing with. In addition, you say that the WD/SI component had 'predetermined' minimum scores. Maybe, maybe not. If you have evidence that they were predetermined, please share. I don't pretend to have all the facts, or very many at all. As to your point about the word 'arbitrary', I am not sure why you are directing this at me, or at least at my post. I made a point in an earlier post to ackowledge that OPM was probably NOT arbitrary. As you call it a serious accusation, I feel it necessary to point out I made no such accusation. As to your final point, I fully agree and wish everyone the best in their quest, including hope2balj and this new class of contestants.
|
|
|
Post by Gaidin on May 18, 2015 16:27:01 GMT -5
I disagree. I am not an expert in testing but I have had to review testing protocols from a legal perspective before. The way I read the text posted by funkyodar above is that the SJT/WS grade is given and they will place people in subgroups based upon percentages. The WD/SI component had predetermined minimum scores. Those minimum scores were not based upon how many candidates it produced but instead ensuring that OPM could certify to any agency that the candidates met not only experience qualifications but also had met a threshold for "judginess" (TM). That means they can easily continue to send lower scoring subgroups through to DC without breaking the rules laid out on the testing because its not a passing score. A passing score means you have sufficient judginess (TM). I think that the easy to grant appeals like Hope2B's above will also start resolving themselves. If someone missed the email saying they got through to DC they will likely get a similar email inviting them on. I am a little concerned everybody keeps tossing the word arbitrary around. That is a really serious accusation to toss at an agency and is usually accompanied by the word capricious. I wouldn't toss it out there without some evidence to back it up. Lastly, please do not interpret this as me saying that anyone doesn't deserve to make the register. I know several people that I met along the trail who I believed had the experience and qualities necessary to be good ALJs. I believe many more people who got cut loose before the end of the journey were also quality candidates. I wish everybody seeking an appeal good luck and I hope you all get a legitimate opportunity for your appeal to be heard. Okay, you may disagree, but I was merely pointing out that the announcement makes it sound pretty final as to the online component as it states that if you are not in the higher scored subgroup, "you will no longer be considered". That sounds like finality to me, and the only prescribed solution for those in that situation - from the announcement - is an appeal. Now we hear differently, and I think that's fine. I am not sure what you are disagreeing with. In addition, you say that the WD/SI component had 'predetermined' minimum scores. Maybe, maybe not. If you have evidence that they were predetermined, please share. I don't pretend to have all the facts, or very many at all. As to your point about the word 'arbitrary', I am not sure why you are directing this at me, or at least at my post. I made a point in an earlier post to ackowledge that OPM was probably NOT arbitrary. As you call it a serious accusation, I feel it necessary to point out I made no such accusation. As to your final point, I fully agree and wish everyone the best in their quest, including hope2balj and this new class of contestants. You were not the only using the word arbitrary by any means but I can't quote more than one person. I wasn't trying to single anyone out. I quoted you to discuss what I believe (guess) are things you got wrong about the differences between SJT phase and the WD phase. I would argue that the substantive differences in the language mean something substantive is different between how the scores will be treated. Regarding my belief about the WD/SI minimum scores being preset that is based upon working with other testing regimes. A preset minimum score is not arbitrary nor capricious by virtue of being established before any candidate is known. Changing it to achieve different results could be considered arbitrary and capricious. Think of the Bar Exam if the passing score changed every time to ensure a certain percentage passed it would be arbitrary because it didn't guarantee a base standard of knowledge. This differs from subgroup scoring if the subgroup scoring was designed to be used to increase the size of the register. In other words they designed the process so that the top 25% is the higher scored subgroup, and each lower 25% is another subgroup that proceeds to testing when the register needs additional names. This way they don't spend all the money testing 4,000 people when they need 1,000. They can hold the remaining subgroups in reserve until the pool gets stale (to many people have left it for whatever reason) and a refresh becomes necessary. OPM and the agencies still have the SI/WD to ensure that their judginess standard is met.
|
|
|
Post by sealaw90 on May 18, 2015 16:45:35 GMT -5
Wow, drive for a day for some conference I have to attend and BAM, the thread goes haywire! First, hope2balj - awesome news and thanks for posting the email. I'm glad you got this one! Whatever nonsense we talked about in regard to "phase 2" testing (SJT/WC/EA) can be found in a few threads from 2013. We cannot provide any more information other than to say read the directions carefully and good luck.
Second, OPM is not being arbitrary in moving the line between scoring 'subgroups'. This is extremely deliberate and they can determine exactly how many folks get to join the higher scoring subgroup to move on to Phase 3(WD/SI in DC). As a matter of fact, they know everyone's score who took phase 2 testing. For example, by including folks who scored 2 points lower back in 2013, they may capture another 900 folks to test in DC. The application materials as written does not lead to a presumption of arbitrariness. However, if I was kicked out at some point in the testing, you better believe I would've made that argument. Capricious too. Incorrectly applied testing criteria, etc.
Third, Pixie is on point with this new addition to the testing process. Although I do worry a little that more folks could now get a higher score than me, I don't think that will happen. For those of us at the bell curve of scores and higher, I don't think that is a great possibility. Many of us will already be hired (if they would just select my d*#m cities!) before the Summer of '15 folks arrive on the register.
Gaidin, I feel a new poll may be warranted - who got an invite to the summer of '15 testing...lolls like next month we may need it!
|
|
|
Post by sealaw90 on May 18, 2015 16:50:41 GMT -5
Ok, just read Gaidin's last post. Exactly right on the subgroup scoring! OPM knew exactly what they were doing, even if ODAR and the rest of us didn't know what they meant by scoring sub groups.
|
|
|
Post by privateatty on May 18, 2015 16:58:48 GMT -5
yeah, 71stretch, that is about what this rollercoaster has amounted to for anyone still waiting. never seen an alj hiring like this one before.. That much is true. I think OPM has figured out the most cost effective way to serve its clients--the agencies.
|
|