: Technical Program Manager, Data Labeling
245 West 17th Street
New York, NY 10011

Technical Program Manager, Human Computation

Location flexible, candidates considered in Seattle, San Francisco, New York, or Boulder. Fully-Remote is also acceptable

Twitter Service is one of the premier scaled support service orgs within Twitter. The services we provide range from enforcement operations, to Human Computation & Localization. Our goal is to provide meaningful and effective services to our customers, be it internal or external. 

The Human Computation team’s mission is to streamline production of high-quality labeled data and context-based insights & measurement, in support of Machine Learning, offline metrics, and ALL other human-in-the-loop initiatives across Twitter. Our work directly impacts the quality and consumption of our most popular products, helping to prevent the abuse and misuse of Twitter, and delighting our ever expanding global user base. 

We’re looking for a dynamic Technical Program Manager who has a proven track record of designing and driving high-impact high-visibility Engineering and Engineering support programs. The right person would have experience driving projects/initiatives for Human Computation/ Data Annotation and working cross-functionally (across Engineering, Product, Research, Legal, Operations) in support of large company objectives. The right person would also have experience working with multiple global staffing vendors, managing scaled contingent business resources, and effectively navigating a challenging, fast-paced, and dynamic daily working environment.

Responsibilities:

  • Independently drive global team goals, roadmap, priorities, and initiatives to ensure support for Machine Learning projects and accurate qualitative measurements.
  • Drive the development of various technical solutions that are essential to running Human Computation effectively and efficiently, at scale. Manage internal and external tooling backlogs.
  • Identify opportunities, define, scope, and get buy-in across cross-functional partners for projects that can leverage human evaluations to improve existing Machine Learning models and research in support of opportunity sizing new features/understanding usage behavior of new launches. 
  • Work directly with Engineers, Product Managers, Researchers, and Data Scientists to define, scope, build, and execute human evaluations in support of different areas of the product.
  • Interface with cross-functional stakeholders across Operations, Policy, Legal, Procurement, and external vendors to produce quarterly roadmaps in support of top-level company goals.
  • Write clear and concise technical documentation and obtain buy-in from leadership across a wide variety of stakeholder teams. 
  • Review and provide feedback on technical design documents from EPDR, with respect to Human Computation’s scope. 
  • Train, oversee, and own accountability for an international and designated contingent team, large-scale and multilingual liquid workforce, and a crowdsourced workforce.
  • Effectively determine contingent staffing needs; implement effective roles, responsibilities, and global team structure.
  • Review and/or process sensitive and potentially disturbing content including but not limited to pornographic images, child pornography, vulgar language, hate speech, violent threats, and other graphic images.

Minimum Qualifications

  • BA or BS in Cognitive Science, Computer Science/Engineering, or equivalent work experience in a related field. 
  • Over 3 years of experience driving large scale Engineering & Engineering-support programs. 
  • Over 2 year direct experience with Human Computation, crowdsourcing, or experimental design.
  • Over 1 year experience with HTML, XML, CSS, JavaScript, and SQL or specialized database query languages.
  • Proven ability to identify and resolve systemic issues in both software and people systems.
  • Excellent written and verbal communication skills, organizational skills, and highly developed interpersonal skills.
  • Active, passionate Twitter user with a strong interest in social media. Passionate interest in Twitter’s products and mission, and about the role machine learning could play towards advancing them.

Prefered Qualifications

  • MA, MS, or PhD Cognitive Science, Computer Science/Engineering, or equivalent work experience in a related field. 
  • Over 5 years of experience driving large scale Engineering & Engineering-support programs. 
  • Over 2 years experience working with large data-sets & exposure to basics of Machine Learning.
  • Over 1 year experience with R/Python and other proprietary data extraction and analysis tools. 
  • Over 2 years experience designing and implementing research experiments.
  • 3-5 years experience managing, directing, and evaluating the work of others.
  • Proficiency in one or more non-English languages.
Full-time