This is an old revision of the document!
Ben's Weekly Updates
January 31st
Completed
Submitted abstract for reliability and fairness audit of ACP models (Manuscript due March 31st)
Finished onboarding: Get access to data section
TODO
Start drafting background information for manuscript
Meet with team to determine goals and plan for the survey/interview
January 24th
Completed
Met with Jonathan and Siyun to get me up to speed on the reliability and fairness audit of Epic
EOL
TODO
Start drafting manuscript/background information, as part of getting up to speed.
Review model card for ACP when ready
Meet with a couple more members of the lab
Finish onboarding: Get access to data section
Personal: Childcare isn't happening again this week (covid) so this'll be the fourth long week in a row!
January 18th
Completed
Explored explainability project. Decided that my research question had pretty much been answered so I'll pivot to a new project. Jason pointed out an interesting paper involving what they call concept bottlenecks:
https://arxiv.org/abs/2007.04612
Finished onboarding: Summary of work streams
TODO
Meet with Jonathan and Siyun to get integrated into the reliability and fairness audit for epic
EOL.
Flesh out research question: What is a good way to perform a reliability and fairness audit of multiple algorithms?
Finish onboarding: Get access to data section
January 10th
Completed
Explored some potential rotation projects and met with team members (Jason, Ethan, and Mars). Potential Projects:
(Scotty) Poking at explainability. The hypothesis is that people will make up a story even if there isn't one and explainability in the affirmative is not helpful. Rather, it should be used in the negative. Run an experiment with something along the lines of ask expert to explain model (using SHAP), rebuild model after some permutation to the data (TBD on details), and ask expert to again explain model.
(Jason) Change CLMBR pretraining objective to masked language modeling and an autoregressive fine-tuning/adaption stage.
(Unknown) On potential rotation projects document, there is “predict who does not need a Chest CT (for PE)?” Has this project been worked on? I don't know the clinical workflow for PE, what data would be used before CT?
Started onboarding
TODO
Finish onboarding
Write summary of 2 work streams
Get access to data section
Choose topic area and specific rotation project and jump in!
I will start explainability project
Compile list of literature
Paragraphs on the hypothesis/problem and why it matters
Start designing experiment
Begin finding application area and experts