|
Post by jimmyjiggles on Oct 31, 2017 7:00:38 GMT -5
Well it would not impact the number of dispos an ALJ does. That would be a better way to put it. Are you hearing less cases because of the writing backlog Pixie? I didn’t think so. We all know that the opposite of what if1 says is true- it takes far less time to pay a case than deny it. Hence the “paying down the backlog” trope. I would be interested in how much less time you think you can put into a case than the 2-3 hours judges do now (including hearing time). No matter what the criteria, you still have to read those records. That takes time. If you are looking at reducing the time a judge looks at the case (or “hoops” the judge must jump though - an interesting statement given that SSDI is probably the easiest area of law to practice) as opposed to reducing the number of applications coming in (or god forbid, increasing staff), you are barking up the wrong tree in re: reducing the backlog. ETA: blaming the backlog of people waiting for hearing on writers is sort of beyond disingenuous since people having decisions written have necessarily already had hearings and ergo one has literally nothing to do with the other. Huh? My point was about processing time, not the number of dispositions. As to your second point, I wasn't aware the writers were being blamed for the backlog of claimants awaiting hearings? That is a non sequitur. Pixie Not to belabor the point, but if1’s argument is that judges spend too much time jumping through hoops. The argument goes that if the criteria were more stringent, judges would spend less time with a case and ergo could do more dispos per year. Thus the judges “processing time” (and here I mean only the time a judge actually spends with the case, as opposed to writers, scts, etc which as far as I know is not a stat currently tracked in CPMS or elsewhere) is directly linked to how many dispos they can do. That’s if1’s argument anyway. It seems to me that a necessary corrolary to that argument would be that the cases also take too much time to write (the explanation of the hoop jumping) and therefore reducing writing time would reduce the backlog. This would be true if judges wrote their own cases (thus reducing the amount of time they had to review and decide other cases), but as we all know that is not true. Writing time does not impact the judges dispos at all. If1’s argument appears to blame writers (or more precisely the onerous requirements of writing a decision) for the backlog because they spend too much time writing a decision, but in fact that has nothing to do with the backlog of people awaiting a hearing, only the relatively small (5% or so) number of folks who have had a hearing but are awaiting a decision.
|
|
|
Post by maquereau on Oct 31, 2017 9:06:36 GMT -5
If I were to receive more draft decisions from the writers and they were better written, I would produce more dispositions yearly (because, for me, editing consumes more time than any other aspect of this job). If the standards were changed such that unfavorable decisions were more easily written - so that even really crappy writers could write them - I would produce more dispositions yearly.
It's that simple. Really.
|
|
|
Post by stevil on Oct 31, 2017 9:40:02 GMT -5
Claimants should be offered the choice of the normal process, or agreeing to an event randomizer that would settle their claim one way or the other immediately. I believe any coin would suffice for flipping.
That said any system of adjudications is only as good as its acceptance by the public. I can't even wrap my head around driverless semis on the road, much less "cognitive" automation for highly personal situations.
|
|
|
Post by maquereau on Oct 31, 2017 9:42:57 GMT -5
Can AI help us to write unfavorable decisions? Right now, the only thing that I think of, because it would be so incredibly difficult to apply a set of style rules, is a system of prompts that asks writers to reconsider odd phrasings, misused vocabulary, incorrect cites, and, possibly, failure to substantiate findings. Plain ol' Grammar Check should catch a lot more than it does now, so there are obvious limits to the ability of technology to assist us in this endeavor.
|
|
|
Post by jimmyjiggles on Oct 31, 2017 10:40:11 GMT -5
If I were to receive more draft decisions from the writers and they were better written, I would produce more dispositions yearly (because, for me, editing consumes more time than any other aspect of this job). If the standards were changed such that unfavorable decisions were more easily written - so that even really crappy writers could write them - I would produce more dispositions yearly. It's that simple. Really. Fair point, and one that has me curious: if you did not have to edit decisions at all- they came back perfect each time either because you have great writers or AI could write decisions perfectly, how many more cases do you think you could do a year? Obviously ALJs vary in their editing time, but this appears to be one area where AI could, in theory, help reduce the backlog. I am wary of AI “reading” the file for ALJs prehearing, simply because if you rely on that there’s no way to know what you missed. AI could perhaps do a synopsis or bookmark/note take, but I’m not sure how much of a time saver that is if you still have to read the file.
|
|
|
Post by bowser on Oct 31, 2017 11:15:11 GMT -5
AI does not need to issue dispositions in the majority of cases to significantly impact the backlog. What percentage of cases that you hear really should have been disposed of prior to a hearing? If even 1%, that is a huge number.
Just quickly off the top of my head, AI could: -nail down address queries for unrepped claimants -identify cases where age categories passed since state agency, which would result in FF Grid as of new age category -identify income after AOD, and invite reps to address prior to hrg -identify cases w/ significant gaps in medical evidence.
AI could also help in organizing the file - having med records all in F, assigning more accurate dates, etc.
AI need not be a cure-all to have a significant beneficial effect.
|
|
|
Post by maquereau on Oct 31, 2017 11:32:34 GMT -5
If I were to receive more draft decisions from the writers and they were better written, I would produce more dispositions yearly (because, for me, editing consumes more time than any other aspect of this job). If the standards were changed such that unfavorable decisions were more easily written - so that even really crappy writers could write them - I would produce more dispositions yearly. It's that simple. Really. Fair point, and one that has me curious: if you did not have to edit decisions at all- they came back perfect each time either because you have great writers or AI could write decisions perfectly, how many more cases do you think you could do a year? Obviously ALJs vary in their editing time, but this appears to be one area where AI could, in theory, help reduce the backlog. I am wary of AI “reading” the file for ALJs prehearing, simply because if you rely on that there’s no way to know what you missed. AI could perhaps do a synopsis or bookmark/note take, but I’m not sure how much of a time saver that is if you still have to read the file. I would guess that I could issue somewhere around 1/3 more decisions per year. I still have lots of time invested in reading the file, making notes, having the hearing, but there would be very substantial time savings on the EDITs, ..... and there would be the savings on my sanity as well.
|
|
|
Post by jimmyjiggles on Oct 31, 2017 17:01:41 GMT -5
Huh? My point was about processing time, not the number of dispositions. As to your second point, I wasn't aware the writers were being blamed for the backlog of claimants awaiting hearings? That is a non sequitur. Pixie Pixie is right, I'm not blaming writers. More writers would also help the backlog, of course. I think everyone agrees on that. But stricter standards to receive benefits would also help. It would also help if SCOTUS stepped in and resolved some of the circuit splits in the level of deference given to ALJs in how they treat certain kinds of evidence. The 2nd and 9th Circuits in particular use something that looks a lot more like preponderance of the evidence than substantial evidence. I think there are many potential tools to reduce processing time and backlog. The only reform I could see that would tighten eligibility and help speed things up would be to eliminate steps 4 and 5. Just make it Listings or bust. Of course since the Listings require medical documentation, there would not really be much of a point in having a hearing except to ask a ME if the claimant equals or meets, which could be done by Roggs anyway. Other things, like ditching the grids, wouldn’t really make much of a dent in the amount of time for reviewing/hearing/deciding the case, at least not that I can see. ODAR/SSA could do a lot without congressional approval, like ditch the regs on evaluating medical opinions and otherwise articulate everything in the decision. Again though that would primarily go to writing time not “decision making time” so while it would be nice in terms of avoiding remands, I’m not sure it would have a huge impact on the number of dispos one can get out, unless you are a heavy editor of decisions.
|
|
|
Post by hopefalj on Oct 31, 2017 17:48:31 GMT -5
Fair point, and one that has me curious: if you did not have to edit decisions at all- they came back perfect each time either because you have great writers or AI could write decisions perfectly, how many more cases do you think you could do a year? Obviously ALJs vary in their editing time, but this appears to be one area where AI could, in theory, help reduce the backlog. I am wary of AI “reading” the file for ALJs prehearing, simply because if you rely on that there’s no way to know what you missed. AI could perhaps do a synopsis or bookmark/note take, but I’m not sure how much of a time saver that is if you still have to read the file. I would guess that I could issue somewhere around 1/3 more decisions per year. I still have lots of time invested in reading the file, making notes, having the hearing, but there would be very substantial time savings on the EDITs, ..... and there would be the savings on my sanity as well. As someone who has stopped concerning himself with editing decisions, I can certainly affirm the savings on the sanity aspect of the job. Can't wait for my agree rate to drop, but frankly, it's unsustainable to have to sit there and lock up every decision or make them even halfway adequate sometimes (not that I have to tell you that).
|
|
|
Post by jimmyjiggles on Nov 1, 2017 11:03:41 GMT -5
The only reform I could see that would tighten eligibility and help speed things up would be to eliminate steps 4 and 5. Just make it Listings or bust. Of course since the Listings require medical documentation, there would not really be much of a point in having a hearing except to ask a ME if the claimant equals or meets, which could be done by Roggs anyway. Other things, like ditching the grids, wouldn’t really make much of a dent in the amount of time for reviewing/hearing/deciding the case, at least not that I can see. ODAR/SSA could do a lot without congressional approval, like ditch the regs on evaluating medical opinions and otherwise articulate everything in the decision. Again though that would primarily go to writing time not “decision making time” so while it would be nice in terms of avoiding remands, I’m not sure it would have a huge impact on the number of dispos one can get out, unless you are a heavy editor of decisions. "Listings or bust" would definitely help. Of course, reps would just start lobbying to get the listings expanded and training claimants to get listing diagnoses from docs (and identifying which docs were pushovers), but you could backstop the latter by writing a regulation saying examining/consulting physicians for state agencies get equal deference with treating physicians. Again, I am not sure why a results oriented approach (i.e. increase denials) should be favored, at least from the perspective of SSA and concerns about the backlog. Make it Listings or bust, and ALJs will be able to do more cases, period. Whether more or less people get paid is sort of beside the point. Getting a diagnosis is only the first step. Most listings have specific criteria that is met through objective testing. So unless these docs are willing to falsify testing (MRIs, cardiac caths, echos, PFTs, etc) for the claimants, then I don’t think that’s much of an issue. Of course the exceptions are the mental listings and others that use the B criteria with their very ill defined and almost totally subjective categories of “marked” and “extreme” (similarly “inability to ambulate effectively” is somewhat wishy-washy). As for the “lobbying power” of the claimants bar, well they have none. Claimants reps are not trial lawyers, and most eek out a living and don’t have tons of cash to throw to a lobbyist. Constituents who are disabled in their individual capacity have more power than NOSSCR, if only because they make for better photo ops and life stories. The real losers in this scenario would be the VEs, who would be totally shut out. MEs, OTOH, would love it.
|
|
|
Post by magisterludi on Nov 1, 2017 19:10:04 GMT -5
OAO DQ AI (sort of)
The Division of Quality (DQ) met its goal to review a computer-generated random sample of at least 3,500 favorable hearing-lev-el decisions, reported on the selective sample of 1,333 medical equivalence decisions (see p. 4, Data Analysis Contribution), and undertook a new selective sample study of hearing-level dis-missals attributed to claimants’ failure to appear at hearings. In addition, as the most frequent user of the Appeals Council’s own-motion review authority, DQ staff assumed responsibility in De-cember 2016 for OAO’s historic “bureau protest” workload (OAO newsletter 6/27/14, p. 4).
|
|
|
Post by magisterludi on Nov 1, 2017 19:11:39 GMT -5
More FY2017 OAO AI
Actions Sampled: In its seventh full fiscal year of operation, DQ completed pre-effectuation review of a computer-generated random sample of 4,765 unappealed hearing office favorable decisions in equal numbers from all the regions, exceeding its goal of processing 3,500 cases to obtain a statistically valid random sample at the regional level. Of the 4,765, the AC declined own-motion review in 4,115 cases (86.36%) allowing their effectuation and AC exercised own-motion review in 650 cases (13.64%).
|
|
|
Post by magisterludi on Nov 1, 2017 19:14:00 GMT -5
HAL at it again
OAO brought the benefits of its data analytics expertise to several areas in FY 2017. It made a substantive contribution to agency guidance to ALJs for their step 3 evaluation of disability claims and continued training for officials inside and outside the agency interested in development of data analytics capabilities. In addition, OAO provided analysts and adjudicators with a new suite of voluntary, in-house developed software tools called INSIGHT for OAO that is designed to aid in the thoroughness, efficiency, and consistency of case review.
|
|
|
Post by magisterludi on Nov 1, 2017 19:15:25 GMT -5
Do special studies use AI?
Data Analysis Contribution to SSR Development:
An OAO Division of Quality special study of ALJ favorable decisions based on medical equivalence to a listing (step 3) found a lack of articulation of how the claims did not meet a listing and how they equaled a listing. Based on OAO’s
data and analysis, the agency added specific guidance to a then-under development Social Security Ruling (SSR 17-
2p). The additional language advised adjudicators to identify the specific listing section involved, articulating how the
record did not meet the requirements of the listed impairment(s), and how the record, including medical expert or
Medical Support Staff evidence, established an impairment of equivalent severity (OAO newsletter 8/25/17).
|
|
|
Post by magisterludi on Nov 1, 2017 19:18:22 GMT -5
INSIGHT is what most folks equate to OHO AI
INSIGHT Tool Deployment: OAO rolled out the webbased tool in August to analysts and adjudicators in all disability,
quality review, and civil action branches. OAO had conducted a successful pilot test of INSIGHT earlier in the calendar year in four branches following about 18 months of in-house development. The INSIGHT tool flags quality issues in hearing-level decisions to assist OAO analysts in preparation of recommendations to the AC and aid AC members in review of those recommendations. INSIGHT relies on natural language processing and artificial intelligence technologies to read the text of hearing decisions and flag potential policy compliance problems or internal inconsistencies within the decisional text. The end goal is improvement in the quality, consistency, and timeliness of Appeals Council adjudication (OAO newsletter 9/22/17).
Hearing-level Assistance: OAO and the Office of Research, Evaluation, and Statistics partnered to produce a Naïve Bayes classifier statistical model to assist ODAR with triaging cases pending at the hearing level. The classifier helps identify pending cases that have characteristics similar to cases that had received fully favorable decisions without a hearing. Redirecting these cases to the ODAR
National Adjudication Team and the SmartMands team frees up hearing schedule slots and ALJ time. Initial results were promising.
|
|
|
Post by magisterludi on Nov 1, 2017 19:22:57 GMT -5
What ACE is doing at the AC Analytics Center of Excellence for its support of OAO natural language processing, data mining, and analytics projects to improve the disability adjudication process.
|
|
|
Post by magisterludi on Nov 1, 2017 19:28:17 GMT -5
Regarding the current CARES Plan - AI type things Proactive Analysis and Triage for Hearings (PATH) – PATH is a new initiative introduced in FY 2017. However, this initiative builds upon successful screening and data analytic tools developed for the SmartMands and National Adjudication Team (NAT) initiatives from the 2016 CARES Plan. PATH also incorporates the robust use of naïve Bayes classification that will identify cases likely for allowance prior to hearing assignment. Through this initiative, we will assign appropriate staff to review and process cases identified through our screening methodologies. We plan to continue developing the PATH methodology in an effort to use this robust analysis at all levels of disability processing. Through PATH, we expect an increase in non-ALJ adjudications (reversals, on-the-record decisions), which will create a significant savings and opening a hearing slot for another case where a hearing is necessary. Our early projections for PATH modeling efforts in March 2017 suggested that approximately 3 percent (about 22,000) of unassigned cases pending at the hearing level could be identified by this model to be appropriately reviewed for a fully favorable decision without a hearing. We continue to monitor the percent of cases that are selected through the PATH model to validate our expected outcomes and will continue to update and improve our triage models as we learn from and incorporate the results of our efforts. Research and Develop a Strategy for Clustering Work Assignments in the Hearings Operation – This CARES initiative is new for FY 2017. We will optimize how cases are assigned to decision writers and other support staff at the hearings level by assigning cases with like characteristics and/or assigning cases based on projected case complexity. We have realized successes in this model at the Appeals Council level, and we expect to increase decisions written while decreasing average wait times. Special Review Cadre – This is a new CARES initiative for FY 2017 that introduces a special hearing unit for fraud/redetermination cases. We will create a new cadre of ALJs to focus on fraud and/or redetermination cases. Through this initiative, we expect to increase efficiencies in processing of fraud and/or redetermination cases. Information Technology Innovations and Investments – At a Glance Modernize ODAR’s Case Processing System Under this category, we are developing a more comprehensive and up-to-date case processing system across ODAR to be integrated into SSA’s overall disability system. This will improve communication between SSA operating components to ensure consistent disability case processing at all levels and reduce infrastructure costs and maintenance. We are also developing and deploying software in pilot phase that uses artificial intelligence and natural language processing (NLP) to automatically scan case files, identify duplicative medical evidence, and suggest those pieces of evidence for removal by the user. This will increase efficiency and quality of support staff and ALJs, resulting in a decrease of average wait time for claimants. With regard to expanding video hearings capacity, the 2016 CARES Plan included plans for replacing or upgrading old equipment and implemented a schedule to replace or upgrade equipment annually. In FY 2017, we made improvements and acquisitions for video hearing equipment, increasing our capacity to hold video hearings by adding over 200 additional units. We are also partnering with other agencies to use available hearing room space in their sites. We are implementing marketing efforts to promote Representative Video Project (RVP) use, in which claimants can attend video hearings in their representatives’ offices using special equipment. Duplicate Identifying Software (DeDoop) - This CARES initiative is new for FY 2017 that will be supported by special anomaly funding. We will develop and pilot software that uses artificial intelligence and NLP to automatically scan case files, identify duplicative medical evidence, and suggest those pieces of evidence for removal by our staff. We will pilot this software in three sites: Mobile, Alabama; Reno, Nevada; and Albany, New York. Upon successful completion of the pilot, we will pursue next steps for broader implementation. We expect to increase efficiency and the quality of support staff and ALJs’ work, resulting in decrease of average wait time. We estimate the time savings will correlate to ALJs being able to hear approximately 8,000 more cases annually. Expand NLP Quality Assurance Tools (Insight) – This initiative is formerly referred to as Proactive Quality and Natural Language Processing in the 2016 CARES Plan. We are testing an inline quality review tool (Insight) that uses NLP to scan a draft decision for language that could result in error. In FY 2017, we will use special anomaly funding to expand Insight into the hearings level from the Appeals Council. We expect to see improvements in inline quality by ensuring legally sufficiency of draft decisions, thus decreasing the number of remanded decisions to the hearings level.
Raise ALJ Scheduling Expectations In March 2017, we announced an increase in the expected number of hearings scheduled per month per ALJ to 50. We expect this increased expectation will have a positive effect on the hearings backlog, while quality review measures will ensure there is no loss in quality, accuracy, or fairness.
|
|
|
Post by JudgeKnot on Nov 2, 2017 8:06:06 GMT -5
Raise ALJ Scheduling Expectations In March 2017, we announced an increase in the expected number of hearings scheduled per month per ALJ to 50. We expect this increased expectation will have a positive effect on the hearings backlog, while quality review measures will ensure there is no loss in quality, accuracy, or fairness. Hmm. They'll increase the expected number of hearings to 600 per year, but quality review measures will ensure the decisions still pass muster. Meanwhile, those ALJs who don't care about quality or quantity will still be ensconced in their chairs and robes for years while management decides if it's worth even trying to coax some improvement out of them. You could say I'm skeptical that these plans will improve productivity, but I'm just an outsider whose situational awareness is pretty much limited to what I read on this board. This kind of makes ALJs seem like a bunch of chicken deboners. "Hey you lackeys, let's pick up the pace, and still maintain the quality we're known for."
|
|
|
Post by maquereau on Nov 2, 2017 13:04:35 GMT -5
Raise ALJ Scheduling Expectations In March 2017, we announced an increase in the expected number of hearings scheduled per month per ALJ to 50. We expect this increased expectation will have a positive effect on the hearings backlog, while quality review measures will ensure there is no loss in quality, accuracy, or fairness. Hmm. They'll increase the expected number of hearings to 600 per year, but quality review measures will ensure the decisions still pass muster. Meanwhile, those ALJs who don't care about quality or quantity will still be ensconced in their chairs and robes for years while management decides if it's worth even trying to coax some improvement out of them. You could say I'm skeptical that these plans will improve productivity, but I'm just an outsider whose situational awareness is pretty much limited to what I read on this board. This kind of makes ALJs seem like a bunch of chicken deboners. "Hey you lackeys, let's pick up the pace, and still maintain the quality we're known for." I wonder why Quality Control measures couldn't ensure no loss in quality, accuracy, or fairness at 150 decisions per ALJ per month? I mean, I'm sure there's great science behind the numbers they pick out now for adjudication goals.
|
|
|
Post by hopefalj on Nov 2, 2017 17:00:48 GMT -5
I am just going to go aheed and state the blantantly obvious: SSA needs to hire between 5,000 to 7,000 more decision writers. ASAP. Period. They can hire all of the ALJs they want but until they hire a sufficient amount of decision writers, the backlog will not necessarily/sufficiently come down. The ratio needs to be no less than 5 to 1. (Not sure what it is like elsewhere, but here its 1 to 1). If there were 5000-7000 decision writers (which would represent an increase of 3000-5000 from the current numbers, I believe), every decision writer would be bored and out of things to do within 2-3 weeks. The current pending is ~73k. Once up to speed, 5k-7k attorneys would knock out the writing backlog and devour any UNWR at a near constant rate with many having nothing to do. Nothing quite so drastic needs to be done. Simply having a 2-1 writer to judge ratio would be more than sufficient to keep up and knock down the writing backlog. It would also be nice if we could all have our own two a la district court judges that we could train how we like things done as well as trust to help us with other issues as well (OTR reviews, possible grid cases, preconference hearings for unrepped claimants, etc.). The problem would be that there are writers no judge would want to write for them, and there are judges no writer would want to work with.
|
|