COSA Micro-Grant Program 2020
# COSA's Pivot during the COVID Pandemic
COSA Micro-Grant Program 2020
In March of 2020, in response to the Covid-19 pandemic, the Clinic for Open Source Arts (COSA) at the University of Denver and The STUDIO for Creative Inquiry at Carnegie Mellon University jointly offered a series of $1500 “micro-grants” to developers building open-source toolkits for artists. Patching the opening left behind when the STUDIO’s May residency was forced to shutter for the season, the micro-grant initiative recognized a need within the open-source community: many developers found themselves with unexpected time on their hands, struggling due to lost income and stalled projects. Golan Levin, the STUDIO’s director, outlined the mission behind this initiative in the initial call for proposals: “We are especially interested in efforts related to improving the accessibility of your projects, including tutorials, learning resources, and improving documentation, but we are open to any notion that will move your project forward for the greater good.”
Levin’s words immediately called to mind a challenge that Ruha Benjamin raised in “Black Mirrors,” a workshop she taught at the School for Poetic Computation’s Code Societies program last January: how, as activists striving for a more conscious and just application of technology, can we stop associating the prospect of change exclusively with top-down endeavors? How can we “reclaim the problem space” through small interventions, breaking apart the limitations that tend to inhere in our assumptions about what constitutes meaningful activism? The past few months have been a formidable time to push for change: from the racial inequities exposed as Covid-19’s casualties disproportionately impact BIPOC to the immense pain, clarity, pressure, and hope that have characterized waves of protest urging us to abolish the carceral state nationwide, Benjamin’s injunction continues to resonate.
What would a “reclamation of the problem space” look like in the realm of open-source technology? The projects that these micro-grants facilitated tell a story about the paramount importance of these small-yet-meaningful interventions. An eclectic series of proposals related to projects in various stages of development, they point towards the fact that the open-source movement is not a monolith, and its objectives and underlying values can’t be summed up in reductive totality. Nevertheless, when analyzing the intentions and outcomes that many grant recipients shared, a handful of themes emerge: accessibility, bias in machine learning, and even the possibility of considering “open source” as an ethos that extends outside the world of programming. If one commonality can be identified, it is the desire to reach towards a more equitable state of the field—a realistic approach that turns a critical eye towards promises of techno-solutionism while carrying out material change to improve our existing digital landscape.
Some of the funded projects, like Luca Damasco’s Py5.js, literally entail the development of open-source software. Py5.js offers a Python-binded version of p5.js, which itself is a browser-based “software sketchbook” designed to make coding available to beginners from a variety of creative backgrounds [1]. Natively, p5.js runs on JavaScript; Damasco’s addition broadens the potential for use, opening new avenues for coders of all skill levels to create 2D media with Python. Cassie Tarakajian’s P5 Editor Compliance project also takes p5.js as its starting point, aiming to expand the traditional p5.js Web Editor interface’s accessibility. In her proposal for the project, Tarakajian explains that new features which have been recently added to the Web Editor are not yet adequately accessibility-tested, outlining the need for a variety of improvements: high-contrast themes, better landmarks, navigability via the keyboard, and updated documentation to ensure that these accessibility features remain up-to-date and are easily maintainable into the future.
As tech educator Chancey Fleet has observed, digital interfaces—much like physical architecture—are often built by default with patterns and barriers that make them inoperable to people with differing accessibility needs. These “dark patterns” or “encoded inhospitalities,” as she refers to them, range from visual interfaces that unintentionally exclude blind or seeing-impaired people to screen-reader unfriendliness and hidden accessibility settings making it challenging for users to enable image descriptions and alt text [2]. A set of best practices for online accessibility—the Web Content Accessibility Guidelines (WCAG)—set forth standards to make the net more widely accessible, but these guidelines are voluntary for developers to follow, generally unenforced and lacking legal teeth. Initiatives like Tarakajian’s, then, which aim to bring existing tools up to WCAG standards, represent a necessary act of community accountability towards disability justice in digital space.
Levin himself has also been working with Processing, the programming language that serves as the raw code beneath users’ JavaScript creations in p5.js. Working with Lingdong Huang and Tatyana Mustakos, Levin created PEmbroider—an open-source Processing library for controlling computerized embroidering machines, meeting an unfulfilled need to make embroidery design files accessible in terms of both cost and code-literacy. Previously, such files were mainly proprietary, with prices beginning around $2500; while cracked versions can be found in certain Python libraries, they remained out of reach for introductory coders. Thanks to its integration with the existing Processing environment, which is lightweight and unintimidating, PEmbroider liberates digital embroidery for a wider range of creative implementations.
A second general trend that emerges across the spring micro-grant recipients’ projects is an interest in machine learning. Yining Shi and Joey Lee refactored the Neural Network feature in ml5 (a browser-based library making Tensorflow’s machine learning tools available to creative coders with varying degrees of experience) in hopes of making the API “as friendly and maintainable as it is efficient and powerful.” This intervention aims to make the software easier to maintain in the long run. Lydia Jessup also set out to widen the ml5 Neural Network’s accessibility by creating a walk-through tutorial on how to prepare a training dataset, aimed at audiences that may not necessarily come from statistics or computer science backgrounds.
Ellen Nickles’ proposal similarly addressed the issue of training models: in her Data and Model Provenance Project, she researched a number of contextual considerations surrounding the pre-trained models that currently exist in the ml5.js library. By collecting more detailed descriptions of their training data and original use context, this information—which Nickles intends to publish on the “Resources” page of the ml5.js website—can help users evaluate the exigencies of existing data with a deeper understanding of its potential quirks and biases. Aidan Nelson’s project similarly seeks to streamline and document existing resources around machine learning: Nelson proposed to document and update the open-source ML-Agents Toolkit, to enable users to train models using reinforcement learning, a specific modality that enables visual feedback to be integrated into the training process, for eventual use online with Tensorflow.js.
Likewise, Caroline Sinders’ Feminist Data Set, a project that started in 2017 and has continued to develop with support from the micro-grant, critically interrogates the nature of data training. Feminist Data Set departs from the basis that ML algorithms must not be taken at face value; the implementations of machine learning that everyday users interact with most often—the black-boxed algorithm behind Facebook’s newsfeed, for instance—are obscure to the general public, their data and processes considered proprietary. This paradigm, as Sinders explains, ought to be a major cause for concern: unknown, consequential, and intimate data shapes the algorithms that, in turn, shape our daily lives. Given that these datasets remain unchecked by public view, their potential to reinforce existing social biases is enormous: algorithmic bias “can amplify harm, bias, misogyny, racism, and white supremacy,” writes Sinders in the project’s description.
Feminist Data Set, then, asks what it would mean “to create technology that acts as harm reduction, that acts actively as critique,” by creating a framework for the development of new “farm-to-table sustainable data” through collecting, structuring, and modeling with a critical eye to each layer of the algorithm-building pipeline. The system is intended to allow for more transparency across the process of harnessing and labeling data, attributing it to specific researchers and creators and collating more contextual information that can help balance the gendered and racial inequities in traditional ML training that lead to algorithmic reproduction of existing social bias.
The question of biased datasets also guides Anton Marini and Rahul Somani’s work, an investigation into the machine learning training process behind their project Synopsis, an open-source tool for “computational cinematography” [3]. While the user-facing tool does not require knowledge of ML development, Marini and Somani tweaked the underlying models to better understand the biases in existing cinematic datasets—that is, the general white, male, eurocentric streak in conventional narrations of film history at large.
Such biases form what Ruha Benjamin describes as “The New Jim Code”: an indifference to social reality that further entrenches existing discrimination by imagining that technology is magically objective [4]. Benjamin notes that indifference to these biases is often even more harmful than overtly malicious racist intent: it creates a system of engineered inequity, “misery for some, monopoly for others,” that dodges critique by presenting itself as innocent and neutral. Benjamin’s line of thought underscores the importance of work like Marini and Somali’s development of Synopsis, highlighting the enduring necessity of building technological tools that uncover the unjust histories that precede them rather than blindly reinforcing them further.
Synopsis also gestures towards a third trend that unites several of the micro-grant recipients: an interest in creating entirely new tools altogether, as opposed to improving and further developing pre-existing open-source initiatives. Natalie Lawhead does precisely this in Electric Zine Maker, a piece of experimental drawing software that facilitates “casual creation” by situating zine-making as an accessible way of working through creative blocks. In stark contrast to “professional-grade” creative tools like the Adobe suite, Lawhead’s project is deliberately shaped with intentions besides “productivity” in mind. Much like paper zines have long stood as a cheap, alternative way of illustrating and disseminating information in opposition to the more established channels of mainstream publishing, the Electronic Zine Maker makes space for creation that progresses fluidly, motivated by a wider range of impulses than simply profit-generation.
Nick Briz’ project, netnet.studio, similarly facilitates creative expression by lessening the intimidation of learning how to write code in the Web’s core languages, HTML/CSS and JavaScript. Briz’s proposal emphasizes that most creative activity online today takes place using proprietary platforms; while they may be lucrative given their ease of use, these platforms rest on an insidious foundation of data collection and monetization, generating profit by predicting and even manipulating their users’ behaviors online and off. netnet positions the Web itself as the original platform for digital expression, structuring accessible tools (Briz describes the space as a “code playground”) for users to improve their code literacy and experience the agency that proprietary platforms deny them.
Char Stiles and Chirag Davé took advantage of a micro-grant to work on a live-coding environment called Code Augmented Reality Live—CARL, for short. CARL offers a rare example of an open-source mobile augmented reality experience; typically, lightweight AR is tied to proprietary engines such as Unity and Spark AR. CARL was designed for “algoraves,” live music events activated by “an immersive and evolving 3D algorithmic world that dances to the music.” [5] The environment allows a programmer to write graphics shaders, which produce real-time audiovisual art that updates continuously on viewers’ phones—a canny inversion of the classic rave culture tenet that phones have no place on the dance floor, offering a lighthearted and accessible introduction to augmented reality and livecoding.
Rebecca Fiebrink’s Sound Control, another original software contribution, uses machine learning to create entirely new musical instruments but requires no knowledge of ML on the user’s end. Instead, it enables them to transpose their movements into sound, harnessing off-the-shelf tools like the humble webcam and computer mouse. With the grant’s support, Fiebrink hopes to prepare video tutorials with a music teacher to better enable other educators and music therapists to integrate the software into their work, along with working to update Sound Control’s web documentation so that parents, teachers, and developers can learn to use the software more effectively. Like Tarakajian’s project, Sound Control positions accessibility at the forefront: Fiebrink hopes that expanded documentation can make the software more usable by parents in the disability community whose children are being taught and cared for from home due to Covid-19, extending the available resources for music therapy and special education.
The shifting educational environment resulting from Covid-19 also informs Neta Bomani and Bomani McClenton’s contribution, a workshop series entitled Out of the Black Box (OBB). Envisioning a more expansive definition of “open-source” that extends beyond software development, McClenton and Bomani created a series of project-based lesson plans to teach primary school children about electronic circuits using conductive clay, drawings, voice-recordings, and letters. OBB started as a workshop series taught in-person in Red Hook, Brooklyn; recognizing the continued need to offer values-based STEAM education to Black and Latinx youth, they published the OBB lesson plans in an “open-source” digital format to make them available regardless of physical proximity.
Similarly, Everest Pipkin undertook a project that reinterprets the meaning of “open-source” more expansively: they created a series of resource lists ranging from “Public Artist Opportunities” to “Tiny Tools” that collect information about national residencies, existing open-source and experimental software projects, and calls for proposals. Pipkin intends to relocate these resource lists from their current home on Google Drive to a personal platform, recognizing the tension inherent in housing the project on a proprietary cloud-based service.
This tension indexes a broader issue in software development that these grants are helping to alleviate: in general, open-source tools can allow for communal creation and shared stewardship, foregrounding values outside the Silicon Valley paradigms of “disruption” and profit-motivation. However, these same tools can feel out-of-reach to audiences without a coding background—some require a working knowledge of GitHub to download and install; others may presuppose at least an introductory knowledge of certain programming languages; yet still others might simply be less widely discussed outside of certain tech-savvy circles. Taeyoon Choi further explains this paradox on his blog, Distributed Web of Care:
“In conferences or demo days, I will often hear someone say that they are trying to ‘democratize’ technology by making tools or resources. For example, ‘I made this open-source tool. Now everyone can do what I do!’ What they are not saying, and the privilege they are ignorant of, is that in order to use the tool you need to be technically fluent, be in an environment where you are accepted as a technical person, and have access to computers and network. In reality, technology, tools and resources alone cannot uphold democracy. Truly democratic societies require ongoing maintenance, stewardship and education.” [6]
As Choi suggests, the value of open-source development is most pronounced when it is approached holistically—when “open-source” is not merely a label, but a methodology that involves valuing labor power, equitably offering space for varied voices, and more broadly prioritizing community and care.
Pipkin’s resource lists, like Bomani and McClenton’s OBB workshops, reflect this holistic understanding insofar as they widen the idea of “open-source” to unravel the distinction between “user” and “programmer” commonly inbuilt in our default tech platforms. As Melanie Hoff writes in “Always Already Programming,” this distinction is essentially unfounded—just as “users” push buttons to navigate the GUI, “programmers” often work with prepackaged functions and variables—even though “programmers” are generally assumed to have more agency in shaping computational systems [7].
Addie Wagenknecht, who developed a project called Router Wars with support from the micro-grant, speaks directly to the importance of upending this distinction and drawing on the open-source movement outside of those activities strictly considered “computation.” “Open-source is about maximizing impact with the least amount of effort,” she explained to me in an interview. While its origins may lie in software development, its core ethos—maximizing impact by drawing on a vast multitude of contributors to realize ideas more equitably—is, according to Wagenknecht, generalizable to other realms, from teaching to activism. A related shift, which Ruha Benjamin advocates, involves broadening our understanding of what constitutes a “tool”: in Race After Technology, she proposes that we must move away from toxic techno-solutionism by devising “abolitionist tools” that include, but are not at all limited to, technological fixes. These necessarily include social tools, not merely hardware and software.
The expanded definition of “open-source” at play in several of these projects, then, offers a timely conclusion: by widening our shared understanding of what it means to “code” beyond the strict boundaries of software development, we can begin to understand the ways in which coded interactions are not merely technological. Social mores, de facto inequalities, and legal protections (or the absence thereof) all constitute codes, which humans both shape and are profoundly shaped by. As the Covid-19 pandemic reveals and exacerbates systemic inequalities, disproportionately affecting marginalized groups along the preexisting fault lines of race, class, and nationality, this invitation to take a more active role in coding kinder and more democratic societies is especially crucial. Beyond simply supporting software developers, therefore, the COSA and STUDIO micro-grants have facilitated a register of critical inquiry that is deeply necessary for these times: they build towards a common understanding that the inherited biases governing proprietary tech are not inevitable or insurmountable—that accessible alternatives are possible, and, imperatively, that it is worthwhile to imagine and implement digital tools that support a more inclusive, thoughtful, and caring society at large.
(Thanks to Adina Glickstein for researching and authoring this text. Information is drawn from COSA documents and interviews with grant recipients where not otherwise cited. Find more of Adina’s work here, or connect with them on Instagram.)
Notes
1.“P5.js overview.” GitHub. Accessed July 8, 2020. https://github.com/processing/p5.js/wiki/p5.js-overview
2.Bogle, Ariel. “The internet thinks you’re a robot, and other ‘dark patterns’ people with disabilities face online.” ABC Science. Accessed July 8, 2020.
3.“Synopsis: Open Source Computational Cinematography.” Accessed July 8, 2020. https://synopsis.video/
4.“Race After Technology: Databite No. 124.” Data & Society. Accessed July 8, 2020. https://datasociety.net/library/databite-no-124-ruha-benjamin/
5.Stiles, Char. “CARL.” Accessed July 8 2020. http://charstiles.com/carl/
6.Choi, Taeyoon. “Oddkins.” Distributed Web of Care. Accessed July 8, 2020. http://distributedweb.care/posts/oddkins/
7.Hoff, Melanie. “Always Already Programming.” GitHub. Accessed July 8 2020. https://gist.github.com/melaniehoff/95ca90df7ca47761dc3d3d58fead22d4