Portfolio item number 1
Short description of portfolio item number 1
Posts by Collection
portfolio
Portfolio item number 2
Short description of portfolio item number 2
preprints
OCR Graph Features for Manipulation Detection in Documents
Preprint, 2020
Detecting manipulations in digital documents is becoming increasingly important for information verification purposes. Due to the proliferation of image editing software, altering key information in documents has become widely accessible. Nearly all approaches in this domain rely on a procedural approach, using carefully generated features and a hand-tuned scoring system, rather than a data-driven and generalizable approach. We frame this issue as a graph comparison problem using the character bounding boxes, and propose a model that leverages graph features using OCR (Optical Character Recognition). Our model relies on a data-driven approach to detect alterations by training a random forest classifier on the graph-based OCR features. We evaluate our algorithm’s forgery detection performance on dataset constructed from real business documents with slight forgery imperfections. Our proposed model dramatically outperforms the most closely-related document manipulation detection model on this task. Download paper here
Recommended citation: Hailey James, Otkrist Gupta, and Dan Raviv. "OCR Graph Features for Manipulation Detection in Documents." arXiv preprint arXiv:2009.05158 (2020). https://arxiv.org/abs/2009.05158
MRZ code extraction from visa and passport documents using convolutional neural networks
Preprint, 2020
Detecting and extracting information from Machine-Readable Zone (MRZ) on passports and visas is becoming increasingly important for verifying document authenticity. However, computer vision methods for performing similar tasks, such as optical character recognition (OCR), fail to extract the MRZ given digital images of passports with reasonable accuracy. We present a specially designed model based on convolutional neural networks that is able to successfully extract MRZ information from digital images of passports of arbitrary orientation and size. Our model achieved 100% MRZ detection rate and 98.36% character recognition macro-f1 score on a passport and visa dataset. Download paper here
Recommended citation: Yichuan Liu, Hailey James, Otkrist Gupta, Dan Raviv. "MRZ code extraction from visa and passport documents using convolutional neural networks." arXiv preprint arXiv:2009.05489 (2020). https://arxiv.org/abs/2009.05489
Participatory Systems for Personalized Prediction
Preprint, 2022
We introduce a family of personalized prediction models called \emph{participatory systems} that improve privacy, fairness and performance user participation and informed consent. Download paper here
Recommended citation: Hailey James, Chirag Nagpal, Katherine Heller, and Berk Ustun. "Participatory Systems for Personalized Prediction." preprint 2022 https://arxiv.org/pdf/2302.03874.pdf
publications
Probabilistic Bias Mitigation in Word Embeddings
Published in NeurIPS Workshop on Human-Centered Machine Learning, 2019
It has been shown that word embeddings derived from large corpora tend to incorporate biases present in their training data. Various methods for mitigating these biases have been proposed, but recent work has demonstrated that these methods hide but fail to truly remove the biases, which can still be observed in word nearest-neighbor statistics. In this work we propose a probabilistic view of word embedding bias. We leverage this framework to present a novel method for mitigating bias which relies on probabilistic observations to yield a more robust bias mitigation algorithm. We demonstrate that this method effectively reduces bias according to three separate measures of bias while maintaining embedding quality across various popular benchmark semantic tasks. Download paper here
Recommended citation: Hailey James, David Alvarez-Melis. "Probabilistic Bias Mitigation in Word Embeddings." Neurips Workshop on Human-Centered Machine Learning (2019). https://arxiv.org/abs/1910.14497
Printing and Scanning Attack for Image Counter Forensics
Published in EURASIP Journal on Image and Video Processing, 2022
Examining the authenticity of images has become increasingly important as manipulation tools become more accessible and advanced. Recent work has shown that while CNN-based image manipulation detectors can successfully identify manipulations, they are also vulnerable to adversarial attacks, ranging from simple double JPEG compression to advanced pixel-based perturbation. In this paper we explore another method of highly plausible attack: printing and scanning. We demonstrate the vulnerability of two state-of-the-art models to this type of attack. We also propose a new machine learning model that performs comparably to these state-of-the-art models when trained and validated on printed and scanned images. Of the three models, our proposed model outperforms the others when trained and validated on images from a single printer. To facilitate this exploration, we create a dataset of over 6,000 printed and scanned image blocks. Further analysis suggests that variation between images produced from different printers is significant, large enough that good validation accuracy on images from one printer does not imply similar validation accuracy on identical images from a different printer.Download paper here
Recommended citation: Hailey James, Otkrist Gupta, and Dan Raviv. "Printing and Scanning Attack for Image Counter Forensics." J Image Video Proc. 2022, 2 (2022). https://doi.org/10.1186/s13640-022-00579-5 https://jivp-eurasipjournals.springeropen.com/articles/10.1186/s13640-022-00579-5
Learning Document Graphs with Attention for Image Manipulation Detection
Published in ICPRAI International Conference on Pattern Recognition and Artificial Intelligence, 2022
Detecting manipulations in images is becoming increasingly important for combating misinformation and forgery. While recent advances in computer vision have lead to improved methods for detecting spliced images, most state-of-the-art methods fail when applied to images containing mostly text, such as images of documents. We propose a deep-learning method for detecting manipulations in images of documents which leverages the unique structured nature of these images in comparison with those of natural scenes. Specifically, we re-frame the classic image splice detection problem as a node classification problem, in which Optical Character Recognition (OCR) bounding boxes form nodes and edges are added according to an text-specific distance heuristic. We propose a system composed of a Variational Autoencoder (VAE)-based embedding algorithm and a graph neural network with attention, trained end-to-end for robust manipulation detection. Our proposed model outperforms both a state-of-the-art image splice detection method and a document-specific method.
Recommended citation: Hailey James, Otkrist Gupta, and Dan Raviv. "Learning Document Graphs with Attention for Image Manipulation Detection." ICPRAI 2022 https://link.springer.com/chapter/10.1007/978-3-031-09037-0_22
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
CS50 Curriculum and Software Intern
Harvard University CS50, Cambridge, Massachusetts, 2017
CS50 is Harvard’s introductory computer science course. I spent the summer of 2017 working with the teaching staff to develop new curriculum and teaching materials for CS50. I also worked on an interview project aimed at increasing diversity in CS at Harvard.
AddisCoder Lecturer and Teaching Assistant
AddisCoder, Addis Ababa, Ethiopia, 2018
AddisCoder is a program created by professor Jelani Nelson to increase access to computer science and offer mentorship to high school students in Ethiopia. With the other teaching staff, I created curriculum, gave lectures, and worked individually with students in summer computer science course for 180 Ethiopian high school students. Following their participation in the program, several students have now been accepted to top universities in computer science.
Private Tutor and Teacher
The Academy at Harvard Square, Cambridge, Massachusetts, 2018
The Academy at Harvard Square is a tutoring center aimed at helping recent immigrants, including children, teenagers, and adults, achieve educational and career goals. I taught English-language, math, science and test preparation courses in addition to one-on-one tutoring sessions.
Patel Teaching Fellow
Harvard University, Cambridge, Massachusetts, 2018
The Patel Fellowship is a university-wide financial award to support equity and inclusion efforts in CS education. I used the fellowship to serve as a specialized teaching fellow for Professor Stuart Shieber, working with struggling students and developing pedagogy to support students of diverse backgrounds. My efforts included teaching functional programming to students in individual and small-group sessions 5-7 days per week, coordinating with teaching staff to ensure equity and inclusion, and using new data management practices to track student progress and validate pedagogical choices.