Sources

This article reports the results of a sequential explanatory mixed-methods approach to explore the impact of online peer-editing using Google Docs and peer-editing in a face-to-face classroom on EFL learners’ academic writing skills. As the study adopted a quasi-experimental design, two intact classes, each with ten EFL learners, attending an IELTS course at a language institute in Sanandaj, Iran, were selected as the participants. IELTS academic writing task 1 and task 2 were used to assess the learners’ academic writing skills, and a semi-structured interview was conducted to explore the learners’ perceptions towards the impact of online peer-editing on academic writing skills. A paired-samples t-test, an independent-samples t-test, and a one-way MANCOVA were used to analyse the quantitative data. The results indicated that peer-editing both through using Google Docs and in the face-to-face classroom significantly developed the learners’ academic writing skills, and that the former outperformed the latter both in the short and long term. Moreover, after controlling for the pre-tests, the two groups performed differently on the four areas of academic writing on both the post- and delayed post-tests. Thematic analysis, used to analyse the qualitative data, highlighted the learners’ positive perceptions towards the impact of online peer-editing on academic writing skills.

Academic libraries offer educational outreach surrounding their technology services available to users. When using those same technology services for internal projects, librarians can better understand the mechanisms behind these technologies, allowing for meaningful outreach efforts in which librarians serve as both primary users and facilitators of use. This paper highlights a case study of leveraging an institutional license to Overleaf, a collaborative LaTeX editor, to rebuild documentation on the use of KiltHub, an institutional repository. This requires a complex mix of text, images, tables, and even videos, and presented a tremendous challenge to format in Microsoft Word and Google Docs. In leveraging the library’s own technologies, the KiltHub team gained a greater understanding of how to use and teach with these platforms, ultimately better serving campus users.

In May 2023 over 90,000 developers responded to our annual survey about how they learn and level up, which tools they're using, and which ones they want.

Result and Artifact Review documentation and badges - V.1.1

While visual programming has had success in some areas such as introductory or domain specific programming, professional developers typically still use a text editor. Designing a visual tool for professionals poses a number of challenges: visualizations must be flexible to support a variety of different tasks, interactions must be fluid to retain productivity, and the visual editing must scale to large software projects. In this paper we introduce Envision, a visual structured code editor that addresses these challenges using an architecture that supports flexible, customizable visualizations, keyboard-centric controls for fluid interaction, and optimizations to ensure good performance for large projects. Experiments with CogTool indicate that Envision's code manipulation techniques are as efficient as those of Eclipse, thus overcoming a major usability barrier for visual programming for professional developers.

This draft document describes the methodology developed by the Cambridge Center for Carbon Credits (4C) for estimating the number of credits to be issued to a project in the tropical moist forest (TMF) biome. It expands on the methodology outlined in (Swinfield and Balmford, 2023). We welcome comments and suggestions in the associated online document at https://tinyurl.com/cowgreport.

Complete documentation and reproducibility of results are important goals for scientific publications. Standard scientific papers, however, usually contain only final results and document only parameters and processing steps that the authors considered important enough. By recording the complete provenance history of the data leading to a publication one can overcome this limitation and allow reproducibility for reviewers, publishers and readers of scientific publications. While the process of capturing provenance information is a growing research subject, here we discuss usually overlooked challenges involved in publishing provenance-complete papers. We report on our experience in preparing and publishing two specific executable papers where we used the VisTrails workflow system to embed full provenance information of the paper and discuss open challenges and issues we encountered.

Version control systems typically rely on a patch language , heuristic patch synthesis algorithms like diff, and three-way merge algorithms . Standard patch languages and merge algorithms often fail to identify conflicts correctly when there are multiple edits to one line of code or code is relocated. This paper introduces Grove, a collaborative structure editor calculus that eliminates patch synthesis and three-way merge algorithms entirely. Instead, patches are derived directly from the log of the developer’s edit actions and all edits commute, i.e. the repository state forms a commutative replicated data type (CmRDT). To handle conflicts that can arise due to code relocation, the core datatype in Grove is a labeled directed multi-graph with uniquely identified vertices and edges. All edits amount to edge insertion and deletion, with deletion being permanent. To support tree-based editing, we define a decomposition from graphs into groves , which are a set of syntax trees with conflicts—including local, relocation, and unicyclic relocation conflicts—represented explicitly using holes and references between trees. Finally, we define a type error localization system for groves that enjoys a totality property, i.e. all editor states in Grove are statically meaningful, so developers can use standard editor services while working to resolve these explicitly represented conflicts. The static semantics is defined as a bidirectional marking system in line with recent work, with gradual typing employed to handle situations where errors and conflicts prevent type determination. We then layer on a unification-based type inference system to opportunistically fill type holes and fail gracefully when no solution exists. We mechanize the metatheory of Grove using the Agda theorem prover. We implement these ideas as the Grove Workbench , which generates the necessary data structures and algorithms in OCaml given a syntax tree specification.

Three text editors were studied using the editor evaluation methodology developed by Roberts and Moran [3, 4]. The results are presented as an extension of the studies by Roberts and Moran, with comparisons to the editors they studied earlier. In addition, supplementary measurements were taken that suggest minor flaws in the Roberts and Moran methodology. Further problems with the methodology are discussed, with an eye toward improving the methodology for future use. Although three significant problems with the methodology are reported, the problems are interesting primarily as lessons for the design of future evaluation methodologies. The Roberts and Moran methodology remains largely useful for the purposes for which it was designed.

An international association advancing the multidisciplinary study of informing systems. Founded in 1998, the Informing Science Institute (ISI) is a global community of academics shaping the future of informing science.

As the availability of digital video becomes commonplace, a shift in application focus will occur from merely accessing video as an independent data stream to embedding video with other multimedia data types into coordinated hypermedia presentations. The migration to embedded video will present new demands on application and support environments: processing of any one piece of video data will depend on how that data relates to other data streams active within the same presentation. This article describes presentation, synchronization, and interaction control issues for manipulating embedded video. First we describe the requirements for embedded video, contrasted against other forms of video use. Next we consider mechanisms for describing and implementing the behavior of embedded-video segments relative to other data items in a document; these relationships form the basis of implementing cooperative control among the events in a presentation. Finally we consider extending the possibilities for tailoring embedded video to the characteristics of the local runtime environment; this forms the basis for adaptive, application-level quality-of-service control of a presentation. In all cases, we describe a mechanism to externalize the behavior of hypermedia presentations containing resource-intensive data requirements so that effective control can be implemented by low-level system facilities based on application-specific requirements. We present our results in terms of the CMIFed authoring/presentation system.

Live programming allows programmers to edit the code of a running program and immediately see the effect of the code changes. This tightening of the traditional edit-compile-run cycle reduces the cognitive gap between program code and execution, improving the learning experience of beginning programmers while boosting the productivity of seasoned ones. Unfortunately, live programming is difficult to realize in practice as imperative languages lack well-defined abstraction boundaries that make live programming responsive or its feedback comprehensible.This paper enables live programming for user interface programming by cleanly separating the rendering and non-rendering aspects of a UI program, allowing the display to be refreshed on a code change without restarting the program. A type and effect system formalizes this separation and provides an evaluation model that incorporates the code update step. By putting live programming on a more formal footing, we hope to enable critical and technical discussion of live programming systems.

An increasingly common characteristic in visual programming languages (VPLs) is level 4 liveness-the constant monitoring of the system state with continuous redisplay as events arrive and computations progress. However, level 4 liveness can be expensive. We present an implementation method that supports level 4 liveness in declarative VPLs, ensuring without "unreasonable" cost that all values on the screen are correctly updated as computations progress. The method is especially well suited for the growing class of declarative VPLs that display continuously time varying calculations and graphics, such as GUI specification VPLs, event based or reactive VPLs, scientific visualization VPLs, or graphical simulation VPLs.

Large Language Models (LLMs) are being increasingly employed in data science for tasks like data preprocessing and analytics. However, data scientists encounter substantial obstacles when conversing with LLM-powered chatbots and acting on their suggestions and answers. We conducted a mixed-methods study, including contextual observations, semi-structured interviews (n=14), and a survey (n=114), to identify these challenges. Our findings highlight key issues faced by data scientists, including contextual data retrieval, formulating prompts for complex tasks, adapting generated code to local environments, and refining prompts iteratively. Based on these insights, we propose actionable design recommendations, such as data brushing to support context selection, and inquisitive feedback loops to improve communications with AI-based assistants in data-science tools.

Direct manipulation interfaces and programmatic systems have distinct and complementary strengths. The former provide intuitive, immediate visual feedback and enable rapid prototyping, whereas the latter enable complex, reusable abstractions. Unfortunately, existing systems typically force users into just one of these two interaction modes. We present a system called Sketch-n-Sketch that integrates programmatic and direct manipulation for the particular domain of Scalable Vector Graphics (SVG). In Sketch-n-Sketch, the user writes a program to generate an output SVG canvas. Then the user may directly manipulate the canvas while the system immediately infers a program update in order to match the changes to the output, a workflow we call live synchronization. To achieve this, we propose (i) a technique called trace-based program synthesis that takes program execution history into account in order to constrain the search space and (ii) heuristics for dealing with ambiguities. Based on our experience with examples spanning 2,000 lines of code and from the results of a preliminary user study, we believe that Sketch-n-Sketch provides a novel workflow that can augment traditional programming systems. Our approach may serve as the basis for live synchronization in other application domains, as well as a starting point for yet more ambitious ways of combining programmatic and direct manipulation.

Programming language design requires making many usability-related design decisions. However, existing HCI methods can be impractical to apply to programming languages: languages have high iteration costs, programmers require significant learning time, and user performance has high variance. To address these problems, we adapted both formative and summative HCI methods to make them more suitable for programming language design. We integrated these methods into a new process, PLIERS, for designing programming languages in a user-centered way. We assessed PLIERS by using it to design two new programming languages. Glacier extends Java to enable programmers to express immutability properties effectively and easily. Obsidian is a language for blockchains that includes verification of critical safety properties. Empirical studies showed that the PLIERS process resulted in languages that could be used effectively by many programmers and revealed additional opportunities for language improvement.

Graphical user interfaces (GUIs) mediate almost all of our interactions with computers, whether it is through web pages, phone apps, or desktop applications. Functional Reactive Programming (FRP) is a promising approach to GUI design. This thesis presents Elm, a concurrent FRP language focused on easily creating responsive GUIs. Elm has two major features: (1) purely functional graphical layout and (2) support for Concurrent FRP. Purely functional graphical layout is a high level framework for working with complex visual components. It makes it quick and easy to create and combine text, images, and video into rich multimedia displays. Concurrent FRP solves some of FRP’s long-standing efficiency problems: global delays and needless recomputation. Together, Elm’s two major features simplify the complicated task of creating responsive and usable graphical user interfaces. This thesis also includes a fully functional compiler for Elm, available at elm-lang.org. This site includes an interactive code editor that allows you to write and compile Elm programs online with no download or install.

We present a user-friendly approach to unifying program creation and execution, based on a notion of "tangible values" (TVs), which are visual and interactive manifestations of pure values, including functions. Programming happens by gestural composition of TVs. Our goal is to give end-users the ability to create parameterized, composable content without imposing the usual abstract and linguistic working style of programmers. We hope that such a system will put the essence of programming into the hands of many more people, and in particular people with artistic/visual creative style.In realizing this vision, we develop algebras for visual presentation and for "deep" function application, where function and argument may both be nested within a structure of tuples, functions, etc. Composition gestures are translated into chains of combinators that act simultaneously on statically typed values and their visualizations.

Both scientists and philosophers of science have recently emphasized the importance of promoting transparency in science. For scientists, transparency is a way to promote reproducibility, progress, and trust in research. For philosophers of science, transparency can help address the value-ladenness of scientific research in a responsible way. Nevertheless, the concept of transparency is a complex one. Scientists can be transparent about many different things, for many different reasons, on behalf of many different stakeholders. This paper proposes a taxonomy that clarifies the major dimensions along which approaches to transparency can vary. By doing so, it provides several insights that philosophers and other science studies scholars can pursue. In particular, it helps address common objections to pursuing transparency in science, it clarifies major forms of transparency, and it suggests avenues for further research on this topic.

Coakley et al. described how they developed Version Aware Word Documents, which is an enhanced document representation that includes a detailed version history that is self-contained and portable. However, they were not able to adopt the unique-ID-based techniques that have been shown to support efficient merging and differencing algorithms. This application note describes how it is possible to adapt existing features of MS Word's OOXML representation to provide a system of unique element IDs suitable for those algorithms. This requires taking over Word's Revision Save ID (RSID) system and also defining procedures for specifying ID values for elements that do not support the RSID mechanism. Important limitations remain but appear surmountable.

This research investigates some of the reasons for the reported difficulties experienced by writers when using editing software designed for structured documents. The overall objective was to determine if there are aspects of the software interfaces which militate against optimal document construction by writers who are not computer experts, and to suggest possible remedies. Studies were undertaken to explore the nature and extent of the difficulties, and to identify which components of the software interfaces are involved. A model of a revised user interface was tested, and some possible adaptations to the interface are proposed which may help overcome the difficulties. The methodology comprised: 1. identification and description of the nature of a ‘structured document’ and what distinguishes it from other types of document used on computers; 2. isolation of the requirements of users of such documents, and the construction a set of personas which describe them; 3. evaluation of other work on the interaction between humans and computers, specifically in software for creating and editing structured documents; 4. estimation of the levels of adoption of the available software for editing structured documents and the reactions of existing users to it, with specific reference to difficulties encountered in using it; 5. examination of the software and identification of any mismatches between the expectations of users and the facilities provided by the software; 6. assessment of any physical or psychological factors in the reported difficulties experienced, and to determine what (if any) changes to the software might affect these. The conclusions are that seven of the twelve modifications tested could contribute to an improvement in usability, effectiveness, and efficiency when writing structured text (new document selection; adding new sections and new lists; identifying key information typographically; the creation of cross-references and bibliographic references; and the inclusion of parts of other documents). The remaining five were seen as more applicable to editing existing material than authoring new text (adding new elements; splitting and joining elements [before and after]; and moving block text).

Computational experiments have become an integral part of the scientific method, but reproducing, archiving, and querying them is still a challenge. The first barrier to a wider adoption is the fact that it is hard both for authors to derive a compendium that encapsulates all the components needed to reproduce a result and for reviewers to verify the results. In this tutorial, we will present a series of guidelines and, through hands-on examples, review existing tools to help authors create of reproducible results. We will also outline open problems and new directions for database-related research having to do with querying computational experiments.

For various reasons, it is important, if not essential, to integrate the computations and code used in data analyses, methodological descriptions, simulations, etc. with the documents that describe and rely on them. This integration allows readers to both verify and adapt the statements in the documents. Authors can easily reproduce them in the future, and they can present the document’s contents in a different medium, e.g. with interactive controls. This paper describes a software framework for authoring and distributing these integrated, dynamic documents that contain text, code, data, and any auxiliary content needed to recreate the computations. The documents are dynamic in that the contents, including figures, tables, etc., can be recalculated each time a view of the document is generated. Our model treats a dynamic document as a master or “source” document from which one can generate different views in the form of traditional, derived documents for different audiences.

Projections of computer languages are tools that help users interact with representations that better fit their needs than plain text. We collected 62 projections from the literature and from a design workshop and found that 60\% of them can be implemented using a table, a graph or a form. However, projections are often hardcoded for specific languages and situations, and in most cases only the developers of a code editor can create or adapt projections, leaving no room for appropriation by their users. We introduce lorgnette, a new framework for letting programmers augment their code editor with projections. We demonstrate five examples that use lorgnette to create projections that can be reused in new contexts. We discuss how this approach could help democratise projections and conclude with future work.

This study explores the outcomes of using Google Docs for collaborative scientific writing and the associated student perceptions. The study included a total of 35 undergraduate students from the educational sciences programme. All students participated in a pre- and post-test design experiment and 10 were randomly selected to attend interviews. The study revealed three key findings: (a) there was a positive correlation between the frequency of interactions with Google Docs (edits, comments, and revisions) and the improvement of collaborative scientific writing; (b) a significant link was observed between the duration dedicated to writing activities and the quality of individual contributions, indicating that extended involvement leads to more substantial input; (c) students perceived Google Docs to be a useful, satisfactory, easy-to-use, engaging, and interactive tool that enhances their productivity in collaborative scientific writing. The results highlight the usefulness of Google Docs for supporting effective collaborative scientific writing in higher education settings.

We describe Chronicle, a new system that allows users to explore document workflow histories. Chronicle captures the entire video history of a graphical document, and provides links between the content and the relevant areas of the history. Users can indicate specific content of interest, and see the workflows, tools, and settings needed to reproduce the associated results, or to better understand how it was constructed to allow for informed modification. Thus, by storing the rich information regarding the document's history workflow, Chronicle makes any working document a potentially powerful learning tool. We outline some of the challenges surrounding the development of such a system, and then describe our implementation within an image editing application. A qualitative user study produced extremely encouraging results, as users unanimously found the system both useful and easy to use.

This paper presents Online Python Tutor, a web-based program visualization tool for Python, which is becoming a popular language for teaching introductory CS courses. Using this tool, teachers and students can write Python programs directly in the web browser (without installing any plugins), step forwards and backwards through execution to view the run-time state of data structures, and share their program visualizations on the web. In the past three years, over 200,000 people have used Online Python Tutor to visualize their programs. In addition, instructors in a dozen universities such as UC Berkeley, MIT, the University of Washington, and the University of Waterloo have used it in their CS1 courses. Finally, Online Python Tutor visualizations have been embedded within three web-based digital Python textbook projects, which collectively attract around 16,000 viewers per month and are being used in at least 25 universities. Online Python Tutor is free and open source software, available at pythontutor.com.

An interactive graphical environment for supporting the development and use of Haskell applications programs is described. The environment, named Vital, is particularly intended for supporting the open-ended, incremental development style often preferred by non-specialist users in which successive steps of program development are motivated and informed by results so far obtained.Significant features of Vital include: the graphical display of data structures in a format defined by a datatype-indexed stylesheet, the way that evaluation of (possibly infinite) values is demand-driven by the action of the user scrolling around an unbounded workspace, and support for copy-and-paste graphical editing of data structures. This latter allows, for example, the user to modify a complex data structure by point-and-click operations, or to create (by functional evaluation) a regular data structure and then edit values or expressions into it. The effect of each editing operation is immediately reflected in the Haskell program source code.

Direct manipulation interfaces provide intuitive and interactive features to a broad range of users, but they often exhibit two limitations: the built-in features cannot possibly cover all use cases, and the internal representation of the content is not readily exposed. We believe that if direct manipulation interfaces were to (a) use general-purpose programs as the representation format, and (b) expose those programs to the user, then experts could customize these systems in powerful new ways and non-experts could enjoy some of the benefits of programmable systems.In recent work, we presented a prototype SVG editor called Sketch-n-Sketch that offered a step towards this vision. In that system, the user wrote a program in a general-purpose lambda-calculus to generate a graphic design and could then directly manipulate the output to indirectly change design parameters (i.e. constant literals) in the program in real-time during the manipulation. Unfortunately, the burden of programming the desired relationships rested entirely on the user.In this paper, we design and implement new features for Sketch-n-Sketch that assist in the programming process itself. Like typical direct manipulation systems, our extended Sketch-n-Sketch now provides GUI-based tools for drawing shapes, relating shapes to each other, and grouping shapes together. Unlike typical systems, however, each tool carries out the user's intention by transforming their general-purpose program. This novel, semi-automated programming workflow allows the user to rapidly create high-level, reusable abstractions in the program while at the same time retaining direct manipulation capabilities. In future work, our approach may be extended with more graphic design features or realized for other application domains.

Concatenative programming inherently expresses composition of sub-tasks of a pipeline, but one uncommonly seen model of this paradigm includes all data values inline within the program. A visual environment for editing and evaluating programs in this model would inherently display state in place, and allow for easy tracing of data flow through the program by watching the values literally move as evaluation steps took place. We propose a visual approach for programming in this style, with function calls and data values interleaved on a single “track”, with specific concrete arguments always adjacent when a function term is evaluated and various affordances for editing, evaluating, and debugging. We then show how extensions to this model to multiple tracks can ease programming in the model and even make available some more inscrutable programming-language features, such as concurrency and effect systems, in a more accessible way.

Programming languages have been trapped in a world of linear textual representations fundamentally unchanged for half a century. Even systems pushing beyond these forms — visual languages, projectional language workbenches, and end-user programming tools — largely ape the strictures of stream-of-bytes compilers and confine themselves to the popular paradigms of conventional textual systems. Instead of recreating what succeeded in textual paradigms, new programming systems should also be exploring what did not — the confounding, confusing, convoluted approaches that fell by the wayside — with the sorts of direct manipulation, spatial connection, and change over time that textual languages could never match; and they should use their control of presentation to let the user choose the right representation for a piece of code in the moment — and change it. We argue that these two points unlock new frontiers for programming systems, and present preliminary explorations to highlight how multiple-representation environments can lower the pressure on more speculative visual paradigms, to encourage more investigation of this underexamined space.

Efforts to push programming beyond static textual code have sought to imbue programming with multiple distinct qualities. One long-acknowledged quality is liveness: providing programmers with in-depth feedback about a program's dynamic behavior as the program is edited. A second quality, long-explored but lacking a shared term of art, is richness: allowing programmers to edit programs though domain-specific representations and interactions rather than solely through text. In this paper, we map the relationship between these two qualities and survey past work that exemplifies them. We observe that systems combining liveness and richness often do so at the cost of an essential quality of traditional programming: composability. We argue that, by combining liveness, richness, and composability, programming systems can better capture the full potential of interactive computation without leaving behind the expressivity of traditional code.

Literate programming tools are used by millions of programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 data scientists to study coding behaviors in a literate programming environment and how data scientists kept track of variants they explored. For participants who tried to keep a detailed history of their experimentation, both informal and formal versioning attempts led to problems, such as reduced notebook readability. During iteration, participants actively curated their notebooks into narratives, although primarily through cell structure rather than markdown explanations. Next, we surveyed 45 data scientists and asked them to envision how they might use their past history in an future version control system. Based on these results, we give design guidance for future literate programming tools, such as providing history search based on how programmers recall their explorations, through contextual details including images and parameters.

Sine the publication of The Psychology of Human-Computer Interaction , the GOMS model has been one of the most widely known theoretical concepts in HCI. This concept has produced severval GOMS analysis techniques that differ in appearance and form, underlying architectural assumptions, and predictive power. This article compares and contrasts four popular variantsof the GOMS family (the Keystroke-Level Model, the original GOMS formulation, NGOMSL, and CPM-GOMS) by applying them to a single task example.

Jupyter Notebooks have been rapidly adopted by researchers across disciplines to perform scientific computing using its interactive and versatile data exploration and analysis capabilities. Furthermore, running Jupyter Notebooks in standalone web application mode has emerged as a fast and more sustainable way for researchers to develop and share scientific applications using a familiar programming language and development environment. However, significant challenges still exist in developing robust and user-friendly scientific applications using Jupyter Notebook due to development complexity and user interface limitations. This paper introduces Japper, a comprehensive Jupyter-based scientific web application development framework. Japper consists of a core toolkit, established best practices, and architectural patterns specifically tailored for the Jupyter ecosystem. It simplifies project initiation and expedites development workflows by providing intuitive interfaces, advanced front-end customization features, and streamlined deployment. Japper has been used in developing several scientific web applications in diverse fields. Two examples are described here to demonstrate its potential as a broadly applicable and effective application development framework.

The choice of an efficient document preparation system is an important decision for any academic researcher. To assist the research community, we report a software usability study in which 40 researchers across different disciplines prepared scholarly texts with either Microsoft Word or LaTeX. The probe texts included simple continuous text, text with tables and subheadings, and complex text with several mathematical equations. We show that LaTeX users were slower than Word users, wrote less text in the same amount of time, and produced more typesetting, orthographical, grammatical, and formatting errors. On most measures, expert LaTeX users performed even worse than novice Word users. LaTeX users, however, more often report enjoying using their respective software. We conclude that even experienced LaTeX users may suffer a loss in productivity when LaTeX is used, relative to other document preparation systems. Individuals, institutions, and journals should carefully consider the ramifications of this finding when choosing document preparation strategies, or requiring them of authors.

This chapter deals with the first part of the Wikipedia talk page analysis. First, the data set, consisting of Wikipedia talk page discourse material, is introduced. Subsequently, this chapter outlines the two-pronged approach taken\&\#160;to the\&\#160;data analysis....

This chapter embeds Wikipedia in society. For one, this chapter presents facts and figures that aim to elucidate on Wikipedia\&\#8217;s prominence in terms of today\&\#8217;s online media landscape. After a general introduction to global internet usage, this chapter...

At the 2008 Computing Research Association Conference at Snowbird, the authors participated in a panel addressing the issue of paper and proposal reviews. This short paper summarizes the panelists' presentations and audience commentary. It concludes with some observations and suggestions on how we might address this issue in the near-term future.

Domain Specific Languages (DSLs) can be implemented as either internal DSL, i.e. essentially a library in a host general-purpose programming language (GPL), or as external DSL which is a stand-alone language unconstrained in its syntax. This choice implies an inherent trade-off between a limited syntactic and representational flexibility (internal DSLs), or an involved integration with GPLs and the need for a full stack of tools from a parser to a code generator (external DSLs). We propose a solution which addresses this problem by representing a subset of a GPL - from simple code patterns to complex API calls - as GUI widgets in a hybrid editor. Our approach relies on matching parametrized patterns against the GPL program, and displaying the matched parts as dynamically rendered widgets. Such widgets can be interpreted as components of an external DSL. Since the source code is serialized as GPL text without annotations, there is no DSL outside the editor - hence the term ‘virtual’ DSL. This solution has several advantages. The underlying GPL and the virtual DSL can be mixed in a compositional way, with zero cost of their integration. The project infrastructure does not need to be adapted. Furthermore, our approach works with mainstream GPLs like Python or JavaScript. To lower the development effort of such virtual DSLs, we also propose an approach to generate patterns and the corresponding text-only GUI widgets from pairs of examples. We evaluate our approach and its implementation on use cases from several domains. A live demo of the system can be accessed at https://puredit.korz.dev/ and the source code with examples at https://github.com/niklaskorz/puredit/.

In the mid-1980s, more than 300 different versions of word processing software existed (Bergin, 2006a, b), but within a decade, Microsoft Word emerged from the pack and became the standard writing tool. MS Word convinced the public to exchange their typewriters for...

Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conversational interfaces cannot explicitly surface the editing changes that they suggest. To give the author more control when editing with an LLM, we present InkSync, an editing interface that suggests executable edits directly within the document being edited. Because LLMs are known to introduce factual errors, Inksync also supports a 3-stage approach to mitigate this risk: Warn authors when a suggested edit introduces new information, help authors Verify the new information’s accuracy through external search, and allow a third party to Audit with a-posteriori verification via a trace of all auto-generated content. Two usability studies confirm the effectiveness of InkSync’s components when compared to standard LLM-based chat interfaces, leading to more accurate and more efficient editing, and improved user experience.

Mathematics is a ubiquitous foundation of science, technology, and engineering. Specific areas of mathematics, such as numeric and symbolic computation or logics, enjoy considerable software support. Working mathematicians have recently started to adopt Web 2.0 environments, such as blogs and wikis, but these systems lack machine support for knowledge organization and reuse, and they are disconnected from tools such as computer algebra systems or interactive proof assistants. We argue that such scenarios will benefit from Semantic Web technology. Conversely, mathematics is still underrepresented on the Web of [Linked] Data. There are mathematics-related Linked Data, for example statistical government data or scientific publication databases, but their mathematical semantics has not yet been modeled. We argue that the services for the Web of Data will benefit from a deeper representation of mathematical knowledge. Mathematical knowledge comprises structures given in a logical language – formulae, statements (e.g. axioms), and theo-ries –, a mixture of rigorous natural language and symbolic notation in documents, application-specific metadata, and discussions about conceptualizations, formalizations, proofs, and (counter-)examples. Our review of vocabularies for representing these structures covers ontologies for mathematical problems, proofs, interlinked scientific publications, scientific discourse, as well as mathematical metadata vocabularies and domain knowledge from pure and applied mathematics. Many fields of mathematics have not yet been implemented as proper Semantic Web ontologies; however, we show that MathML and OpenMath, the standard XML-based exchange languages for mathematical knowledge, can be fully integrated with RDF representations in order to contribute existing mathematical knowledge to the Web of Data. We conclude with a roadmap for getting the mathematical Web of Data started: what datasets to publish, how to interlink them, and how to take advantage of these new connections.

Computational notebooks such as Jupyter are now used by millions of data scientists, machine learning engineers, and computational researchers to do exploratory and end-user programming. In recent years, dozens of different notebook systems have been developed across academia and industry. However, we still lack an understanding of how their individual designs relate to one another and what their tradeoffs are. To provide a holistic view of this rapidly-emerging landscape, we performed, to our knowledge, the first comprehensive design analysis of dozens of notebook systems. We analyzed 60 notebooks (16 academic papers, 29 industry products, and 15 experimental/R\&D projects) and formulated a design space that succinctly captures variations in system features. Our design space covers 10 dimensions that include diverse ways of importing data, editing code and prose, running code, and publishing notebook outputs. We conclude by suggesting ways for researchers to push future projects beyond the current bounds of this space.

The Open Science [OS] movement aims to foster the wide dissemination, scrutiny and re-use of research components for the good of science and society. This Element examines the role played by OS principles and practices within contemporary research and how this relates to the epistemology of science. After reviewing some of the concerns that have prompted calls for more openness, it highlights how the interpretation of openness as the sharing of resources, so often encountered in OS initiatives and policies, may have the unwanted effect of constraining epistemic diversity and worsening epistemic injustice, resulting in unreliable and unethical scientific knowledge. By contrast, this Element proposes to frame openness as the effort to establish judicious connections among systems of practice, predicated on a process-oriented view of research as a tool for effective and responsible agency. This title is also available as Open Access on Cambridge Core.

Patchwork is a research project about version control software for writers, developers, and other creatives. This lab notebook contains snippets of our prototypes and findings.

Design Space Analysis is an approach to representing design rationale. It uses a semiformal notation, called QOC (Questions, Options, and Criteria), to represent the design space around an artifact. The main constituents of QOC are Questions 54identifying key design issues, Options providing possible answers to the Questions, and Criteria for assessing and comparing the Options. Design Space Analysis also takes account of justifications for the design (and possible alternative designs) that reflect considerations such as consistency, models and analogies, and relevant data and theory. A Design Space Analysis does not produce a record of the design process but is instead a coproduct of design and has to be constructed alongside the artifact itself. Our work is motivated by the notion that a Design Space Analysis will repay 55the investment in its creation by supporting both the original process of design and subsequent work on redesign and reuse by (a) providing an explicit representation to aid reasoning about the design and about the consequences of changes to it and (b) serving as a vehicle for communication, for example, among members of the design team or among the original designers and later maintainers of a system. This article describes the elements of Design Space Analysis and illustrates them by analyses of existing designs and by empirical studies of the concepts and arguments used by designers during design discussions. The aim here is to articulate the nature of the QOC representation. More extensive explorations of processes for creating and using it can be found elsewhere (e.g., Buckingham Shum, 1995 [chapter 6 in this book]; MacLean, Bellotti, \& Shum, 1993; McKerlie \& MacLean, 1993, 1994).

A concern with “embodied action” has informed both the analysis of everyday action through technologies and also suggested ways of designing innovative systems. In this article, we consider how these two programs, the analysis of everyday embodied interaction on the one hand, and the analysis of technically-mediated embodied interaction on the other, are interlinked. We draw on studies of everyday interaction to reveal how embodied conduct is embedded in the environment. We then consider a collaborative technology that attempts to provide a coherent way of presenting life-sized embodiments of participants alongside particular features of the environment. These analyses suggest that conceptions of embodied action should take account of the interactional accomplishment of activities and how these are embedded in the material environment.

The anywidget project provides a specification and toolset for portable and reusable web-based widgets in interactive computing environments (Figure 1). First, it defines a standard for widget front-end code based on the web browser’s native module system. Second, it provides tools to author, distribute, and execute these modules across web-based computing platforms. Since its release in February 2023, anywidget has steadily gained adoption. As of October 2024, nearly 100 new widgets have been created or ported to anywidget and published to the Python Package Index (PyPI), along with many standalone scripts and notebooks. These tools cover general-purpose visualization libraries (Heer \& Moritz, 2024; Lekschas \& Manz, 2024) as well as notebook integrations for applications in biology (Mark S. Keller et al., 2021; Manz et al., 2022, 2023; Manz, Lekschas, et al., 2024), mapping (Barron, 2024), astronomy (Boch \& Desroziers, 2020), and education (Warmerdam, 2024). Anywidget has also been integrated into popular visualization libraries like Altair (VanderPlas et al., 2018), enhancing interactivity in notebooks and deepening user engagement with visualizations and code.

Documents such as presentations, instruction manuals, and research papers are disseminated using various file formats, many of which barely support the incorporation of interactive content. To address this lack of interactivity, we present Chameleon, a system-wide tool that combines computer vision algorithms used for image identification with an open database format to allow for the layering of dynamic content. Using Chameleon, static documents can be easily upgraded by layering user-generated interactive content on top of static images, all while preserving the original static document format and without modifying existing applications. We describe the development of Chameleon, including the design and evaluation of vision-based image replacement algorithms, the new document-creation pipeline as well as a user study evaluating Chameleon.

A dynamic language promotes ease of use through flexible typing, a focus on high-level programming, and by streamlining the edit-compile-debug cycle. Live languages go beyond dynamic languages with more ease of use features. A live language supports live programming that provides programmers with responsive and continuous feedback about how their edits affect program execution. A live language is also based on high-level constructs such as declarative rules so that programmers can write less code. A live language could also provide programmers with responsive semantic feedback to enable time-saving services such as code completion. This paper describes the design of a textual live language that is based on reactive data-flow values known as signals and dynamic inheritance. Our language, SuperGlue, supports live programming with responsive semantic feedback, which we demonstrate with a working prototype.

Structure editors have long promised to facilitate a continuous dialogue between programmer and system-one uninterrupted by syntax errors, such that vital program analyses and editor services are always available. Unfortunately, structure editors are notoriously slow or difficult to use, particularly when it comes to modifying existing code. Prior designs often struggle to resolve the tension between maintaining a program's hierarchical structure and supporting the editing affordances expected of its linear projection. We propose the paradigm of gradual structure editing, which mitigates this tension by allowing for temporary disassembly of hierarchical structures as needed for text-like editing, while scaffolding these interactions by generating syntactic obligations that, once discharged, guarantee proper reassembly. This paper contributes the design and evaluation of a gradual structure editor called teen tylr. We conducted a lab study comparing teen tylr to a text editor and a traditional structure editor on structurally complex program editing tasks, and found that teen tylr helped resolve most usability problems we identified in prior work, though not all, while achieving competitive performance with text editing on most tasks. We conclude with a discussion of teen tylr's remaining limitations and design implications for future code editors and parsers.

This adds many technical details to the Humanist Edition.

Text editing is powerful, but some types of expressions are more naturally represented and manipulated graphically. Examples include expressions that compute colors, music, animations, tabular data, plots, diagrams, and other domain-specific data structures. This paper introduces live literals, or livelits, which allow clients to fill holes of types like these by directly manipulating a user-defined GUI embedded persistently into code. Uniquely, livelits are compositional: a livelit GUI can itself embed spliced expressions, which are typed, lexically scoped, and can in turn embed other livelits. Livelits are also uniquely live: a livelit can provide continuous feedback about the run-time implications of the client’s choices even when splices mention bound variables, because the system continuously gathers closures associated with the hole that the livelit is filling. We integrate livelits into Hazel, a live hole-driven programming environment, and describe case studies that exercise these novel capabilities. We then define a simply typed livelit calculus, which specifies how livelits operate as live graphical macros. The metatheory of macro expansion has been mechanized in Agda.

This paper develops a dynamic semantics for incomplete functional programs, starting from the static semantics developed in recent work on Hazelnut. We model incomplete functional programs as expressions with holes, with empty holes standing for missing expressions or types, and non-empty holes operating as membranes around static and dynamic type inconsistencies. Rather than aborting when evaluation encounters any of these holes as in some existing systems, evaluation proceeds around holes, tracking the closure around each hole instance as it flows through the remainder of the program. Editor services can use the information in these hole closures to help the programmer develop and confirm their mental model of the behavior of the complete portions of the program as they decide how to fill the remaining holes. Hole closures also enable a fill-and-resume operation that avoids the need to restart evaluation after edits that amount to hole filling. Formally, the semantics borrows machinery from both gradual type theory (which supplies the basis for handling unfilled type holes) and contextual modal type theory (which supplies a logical basis for hole closures), combining these and developing additional machinery necessary to continue evaluation past holes while maintaining type safety. We have mechanized the metatheory of the core calculus, called Hazelnut Live, using the Agda proof assistant. We have also implemented these ideas into the Hazel programming environment. The implementation inserts holes automatically, following the Hazelnut edit action calculus, to guarantee that every editor state has some (possibly incomplete) type. Taken together with this paper's type safety property, the result is a proof-of-concept live programming environment where rich dynamic feedback is truly available without gaps, i.e. for every reachable editor state.

Large Language Models (LLMs) have shown the potential to be valuable teaching tools, with the potential of giving every student a personalized tutor. However, one challenge with using LLMs to learn new concepts is that when learning a topic in an unfamiliar domain, it can be difficult to know what questions to ask. Further, language models do not always encourage "active learning" where students can test and assess their understanding. In this paper, we propose ways to combine large language models with "traditional" learning materials (like e-books) to give readers the benefits of working with LLMs (the ability to ask personally interesting questions and receive personalized answers) with the benefits of a traditional e-book (having a structure and content that is pedagogically sound). This work shows one way that LLMs have the potential to improve learning materials and make personalized programming education more accessible to a broader audience.

Empirically analyzing empirical evidence One of the central goals in any scientific endeavor is to understand causality. Experiments that seek to demonstrate a cause/effect relation most often manipulate the postulated causal factor. Aarts et al. describe the replication of 100 experiments reported in papers published in 2008 in three high-ranking psychology journals. Assessing whether the replication and the original experiment yielded the same result according to several criteria, they find that about one-third to one-half of the original findings were also observed in the replication study. Science , this issue 10.1126/science.aac4716 , A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired. , INTRODUCTION Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error. RATIONALE There is concern about the rate and predictors of reproducibility, but limited evidence. Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results. Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding and is the means of establishing reproducibility of a finding with new data. We conducted a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of psychological science. RESULTS We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. There is no single standard for evaluating replication success. Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects ( M r = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects ( M r = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results ( P {\textless} .05). Thirty-six percent of replications had significant results; 47\% of original effect sizes were in the 95\% confidence interval of the replication effect size; 39\% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68\% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams. CONCLUSION No single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here. Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know. Original study effect size versus replication effect size (correlation coefficients). Diagonal line represents replication effect size equal to original effect size. Dotted line represents replication effect size of 0. Points below the dotted line were effects in the opposite direction of the original. Density plots are separated by significant (blue) and nonsignificant (red) effects. , Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47\% of original effect sizes were in the 95\% confidence interval of the replication effect size; 39\% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68\% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.

Editing (e.g., editing conceptual diagrams) is a typical office task that requires numerous tedious GUI operations, resulting in poor interaction efficiency and user experience, especially on mobile devices. In this paper, we present a new type of human-computer collaborative editing tool (CET) that enables accurate and efficient editing with little interaction effort. CET divides the task into two parts, and the human and the computer focus on their respective specialties: the human describes high-level editing goals with multimodal commands, while the computer calculates, recommends, and performs detailed operations. We conducted a formative study (N = 16) to determine the concrete task division and implemented the tool on Android devices for the specific tasks of editing concept diagrams. The user study (N = 24 + 20) showed that it increased diagram editing speed by 32.75\% compared with existing state-of-the-art commercial tools and led to better editing results and user experience.

Objectives The existing model of academic writing and publishing in medical research has not strayed far from its correspondence-based letter writing origins. Authors frequently complain that this system is out of date and restrictive. Currently, articles lack transparency - it is difficult to fully and concisely explain complex analyses without presenting full code and dataset structure. Analyses are also fixed and limited - should readers wish to test assumptions, conduct additional tests, or amalgamate study data with other datasets, they are limited to that which is published, or to contacting authors for datasets and code (often unsuccessfully).Literate programming is an approach which allows manuscripts to be more transparent, reproducible, and interactive. There are free, open-source tools (e.g. RMarkdown and Pweave) which allow entire research manuscripts to be generated end-to-end in one continuous interwoven block, with live code which runs upon opening the document, which can be compiled into .pdf or HTML. Method This abstract demonstrates how manuscripts can be written using the literate programming model. Using a cleaned, publicly available dataset taken from Vickers’ 2006 paper (Vickers Trials 2006, 7:15), we use the knitr, rmarkdown, and rticles packages in RStudio, to write a fully transparent example trial results paper. This paper is compiled into .pdf format using the PLOS LaTeX template (Public Library of Science 2018; https://journals.plos.org/plosone/s/latex), a commonly used manuscript template which is under Creative Commons licence. Results The resulting . pdf manuscript has a format familiar to readers, yet has increased transparency and interactivity, as the code and dataset are distributed with, and are integrated in, the manuscript. The script can be therefore be downloaded and altered to test assumptions, derive values not presented in the paper (to be used in meta-analyses, for example), and reproduce results.The compiled paper will be available at the author’s GitHub repository: https://github.com/mattyjparkes/EBMLive2019-example-paper Conclusions These ‘living papers’, whereby code, data, and interpretation are all interwoven into one live-compiled document has numerous applications for meta-level analyses, and a significantly greater level of transparency. They circumvent the limitations discussed with the current static publishing model, allowing readers to interact and unpick papers in a way that is currently not possible with manuscripts that are divorced from their datasets and analysis code.

Python offers basic facilities for interactive work and a comprehensive library on top of which more sophisticated systems can be built. The IPython project provides on enhanced interactive environment that includes, among other features, support for data visualization and facilities for distributed and parallel computation

Recent availability of data about writing processes at keystroke-granularity has enabled research on the evolution of document writing. A natural task is to develop systems that can actually show this data, that is, user interfaces that transform the data of the process of writing –today a black box– into intelligible forms. On this line, we propose a data structure that captures a document’s fine-grained history and an organic visualization that serves as an interface to it. We evaluate a proof-of-concept implementation of the system through a pilot study using documents written by students at a public university. Our results are promising and reveal facets such as general strategies adopted, local edition density and hierarchical structure of the final text.

Macintosh-style interaction design has reached its limits. A new paradigm, called results-oriented UI, might well be the way to empower users in the future.

Various programming tools, languages, and environments give programmers the impression of changing a program while it is running. This experience of liveness has been discussed for over two decades and a broad spectrum of research on this topic exists. Amongst others, this work has been carried o...

Mergers and acquisitions pose complex organizational and technical challenges. A user-centered approach can help ensure the success of a merger. By prioritizing the management of user expectations, by addressing change aversion and by ensuring users feel heard, a merger cannot only be made more successful, but also be perceived as a positive change by end users. This case study describes and discusses the user-centered approach taken to successfully merge two established online LaTeX editors: Overleaf and ShareLaTeX. A mixed methods approach was used to simultaneously ensure broad coverage of the user base (quantitative; relevant because these were established products) and also in-depth understanding of their issues, opinions and motivations (qualitative; relevant to making informed design decisions). This user-centered approach was successful in helping the company to set a path for the merger, to adapt in the face of new data, and to create the foundation for sustained growth.

"Digital fluency" should mean designing, creating, and remixing, not just browsing, chatting, and interacting.

This paper presents a methodology for evaluating text editors on several dimensions: the time it takes experts to perform basic editing tasks, the time experts spend making and correcting errors, the rate at which novices learn to perform basic editing tasks, and the functionality of editors over more complex tasks. Time, errors, and learning are measured experimentally; functionality is measured analytically; time is also calculated analytically. The methodology has thus far been used to evaluate nine diverse text editors, producing an initial database of performance results. The database is used to tell us not only about the editors but also about the users—the magnitude of individual differences and the factors affecting novice learning.

We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking while telling stories. It emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study with our prototype shows that the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

Background : “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods : Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results : This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions : I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

We present a new computing environment for authoring mixed natural and computer language documents. In this environment a single hierarchically-organized plain text source file may contain a variety of elements such as code in arbitrary programming languages, raw data, links to external resources, project management data, working notes, and text for publication. Code fragments may be executed in situ with graphical, numerical and textual output captured or linked in the file. Export to LATEX, HTML, LATEX beamer, DocBook and other formats permits working reports, presentations and manuscripts for publication to be generated from the file. In addition, functioning pure code files can be automatically extracted from the file. This environment is implemented as an extension to the Emacs text editor and provides a rich set of features for authoring both prose and code, as well as sophisticated project management capabilities.

Computational notebooks are commonly used for iterative workflows, such as in exploratory data analysis. This process lends itself to the accumulation of old code and hidden state, making it hard for users to reason about the lineage of, e.g., plots depicting insights or trained machine learning models. One way to reason about code used to generate various notebook data artifacts is to compute a program slice, but traditional static approaches to slicing can be both inaccurate (failing to contain relevant code for artifacts) and conservative (containing unnecessary code for an artifacts). We present nbslicer, a dynamic slicer optimized for the notebook setting whose instrumentation for resolving dynamic data dependencies is both bolt-on (and therefore portable) and switchable (allowing it to be selectively disabled in order to reduce instrumentation overhead). We demonstrate Nbslicer's ability to construct small and accurate backward slices (i.e., historical cell dependencies) and forward slices (i.e., cells affected by the "rerun" of an earlier cell), thereby improving reproducibility in notebooks and enabling faster reactive re-execution, respectively. Comparing nbslicer with a static slicer on 374 real notebook sessions, we found that nbslicer filters out far more superfluous program statements while maintaining slice correctness, giving slices that are, on average, 66\% and 54\% smaller for backward and forward slices, respectively.

JavaScript visualization libraries are widely used to create online data visualizations but provide limited access to their information for screen-reader users. Building on prior findings about the experiences of screen-reader users with online data visualizations, we present VoxLens, an open-source JavaScript plug-in that—with a single line of code—improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach. Specifically, VoxLens enables screen-reader users to obtain a holistic summary of presented information, play sonified versions of the data, and interact with visualizations in a “drill-down” manner using voice-activated commands. Through task-based experiments with 21 screen-reader users, we show that VoxLens improves the accuracy of information extraction and interaction time by 122\% and 36\%, respectively, over existing conventional interaction with online data visualizations. Our interviews with screen-reader users suggest that VoxLens is a “game-changer” in making online data visualizations accessible to screen-reader users, saving them time and effort.

This paper reflects on experiences designing, developing, and working with users of a variety of interactive computer systems. The authors propose, based on these experiences, that the cause of a number of unexpected difficulties in human-computer interaction lies in users' unwillingness or inability to make structure, content, or procedures explicit. Besides recounting experiences with system use, this paper discusses why users reject or circumvent formalisms which require such explicit expression, and suggests how system designers can anticipate and compensate for problems users have in making implicit aspects of their tasks explicit. The authors propose computational approaches that address this problem, including incremental and system-assisted formalization mechanisms and methods for recognizing and using undeclared structure; they also propose non-computational solutions that involve designers and users reaching a shared understanding of the task situation and the methods that motivate the formalisms. This paper poses that, while it is impossible to remove all formalisms from computing systems, system designers need to match the level of formal expression entailed with the goals and situation of the users -- a design criteria not commonly mentioned in current interface design.

Visual languages have been developed to help new programmers express algorithms easily. They also help to make experienced programmers more productive by simplifying the organization of a program through the use of visual representations. However, visual languages have not reached their full potential because of several problems including the following: difficulty of producing visual representations for the more abstract computing constructs; the lack of adequate computing power to update the visual representations in response to user actions; the immaturity of the subfield of visual programming and need for additional breakthroughs and standardization of existing mechanisms. Visualization of Vision Algorithms (VIVA) is a proposed visual language for image processing. Its main purpose is to serve as an effective teaching tool for students of image processing. Its design also takes account of several secondary goals, including the completion of a software platform for research in human/image interaction, the creation of a vehicle for studying algorithms and architectures for parallel image processing, and the establishment of a presentation medium for image-processing algorithms.

Liveness in programming environments generally refers to the ability to modify a running program. Liveness is one form of a more general class of behaviors by a programming environment that provide information to programmers about what they are constructing. This paper gives a brief historical perspective on liveness and proposes an extension of a hierarchy given in 1990, to now account for even more powerful execution-oriented tools for programmers. In addition, while liveness concerns the timeliness of execution feedback, considering a broader array of forms of feedback is helpful both in better understanding liveness and in designing ever more powerful development tools.

A modern biomedical research project can easily contain hundreds of analysis steps and lack of reproducibility of the analyses has been recognized as a severe issue. While thorough documentation enables reproducibility, the number of analysis programs used can be so large that in reality reproducibility cannot be easily achieved. Literate programming is an approach to present computer programs to human readers. The code is rearranged to follow the logic of the program, and to explain that logic in a natural language. The code executed by the computer is extracted from the literate source code. As such, literate programming is an ideal formalism for systematizing analysis steps in biomedical research. We have developed the reproducible computing tool Lir (literate, reproducible computing) that allows a tool-agnostic approach to biomedical data analysis. We demonstrate the utility of Lir by applying it to a case study. Our aim was to investigate the role of endosomal trafficking regulators to the progression of breast cancer. In this analysis, a variety of tools were combined to interpret the available data: a relational database, standard commandline tools, and a statistical computing environment. The analysis revealed that the lipid transport related genes LAPTM4B and NDRG1 are coamplified in breast cancer patients, and identified genes potentially cooperating with LAPTM4B in breast cancer progression. Our case study demonstrates that with Lir, an array of tools can be combined in the same data analysis to improve efficiency, reproducibility, and ease of understanding. Lir is an open-source software available at github.com/borisvassilev/lir.

When teams of data scientists collaborate on computational notebooks, their discussions often contain valuable insight into their design decisions. These discussions not only explain analysis in the current notebook but also alternative paths, which are often poorly documented. However, these discussions are disconnected from the notebooks for which they could provide valuable context. We propose Callisto, an extension to computational notebooks that captures and stores contextual links between discussion messages and notebook elements with minimal effort from users. Callisto allows notebook readers to better understand the current notebook content and the overall problem-solving process that led to it, by making it possible to browse the discussions and code history relevant to any part of the notebook. This is particularly helpful for onboarding new notebook collaborators to avoid misinterpretations and duplicated work, as we found in a two-stage evaluation with 32 data science students.

Real-time collaborative editing in computational notebooks can improve the efficiency of teamwork for data scientists. However, working together through synchronous editing of notebooks introduces new challenges. Data scientists may inadvertently interfere with each others' work by altering the shared codebase and runtime state if they do not set up a social protocol for working together and monitoring their collaborators' progress. In this paper, we propose a real-time collaborative editing model for resolving conflict edits in computational notebooks that introduces three levels of edit protection to help collaborators avoid introducing errors to both the program source code and changes to the runtime state.

Computational notebooks, such as Jupyter Notebook, have become data scientists’ de facto programming environments. Many visualization researchers and practitioners have developed interactive visualization tools that support notebooks, yet little is known about the appropriate design of these tools. To address this critical research gap, we investigate the design strategies in this space by analyzing 163 notebook visualization tools. Our analysis encompasses 64 systems from academic papers and 105 systems sourced from a pool of 55k notebooks containing interactive visualizations that we obtain via scraping 8.6 million notebooks on GitHub. Through this study, we identify key design implications and trade-offs, such as leveraging multimodal data in notebooks as well as balancing the degree of visualization-notebook integration. Furthermore, we provide empirical evidence that tools compatible with more notebook platforms have a greater impact. Finally, we develop SuperNOVA, an open-source interactive browser to help researchers explore existing notebook visualization tools. SuperNOVA is publicly accessible at: https://poloclub.github.io/supernova/.

Blocks-based programming tools are becoming increasingly common in high-school introductory computer science classes. Such contexts are quite different than the younger audience and informal settings where these tools are more often used. This paper reports findings from a study looking at how high school students view blocks-based programming tools, what they identify as contributing to the perceived ease-of-use of such tools, and what they see as the most salient differences between blocks-based and text-based programming. Students report that numerous factors contribute to making blocks-based programming easy, including the natural language description of blocks, the drag-and-drop composition interaction, and the ease of browsing the language. Students also identify drawbacks to blocks-based programming compared to the conventional text-based approach, including a perceived lack of authenticity and being less powerful. These findings, along with the identified differences between blocks-based and text-based programming, contribute to our understanding of the suitability of using such tools in formal high school settings and can be used to inform the design of new, and revision of existing, introductory programming tools.

A new wave of collaborative robots designed to work alongside humans is bringing the automation historically seen in large-scale industrial settings to new, diverse contexts. However, the ability to program these machines often requires years of training, making them inaccessible or impractical for many. This paper rethinks what robot programming interfaces could be in order to make them accessible and intuitive for adult novice programmers. We created a block-based interface for programming a one-armed industrial robot and conducted a study with 67 adult novices comparing it to two programming approaches in widespread use in industry. The results show participants using the block-based interface successfully implemented robot programs faster with no loss in accuracy while reporting higher scores for usability, learnability, and overall satisfaction. The contribution of this work is showing the potential for using block-based programming to make powerful technologies accessible to a wider audience.

It was the most radical computer dream of the hacker era. Ted Nelson's Xanadu project was supposed to be the universal, democratic hypertext library that would help human life evolve into an entirely new form.

We propose a new approach to the visualization design and communication process, literate visualization, based upon and extending, Donald Knuth's idea of literate programming. It integrates the process of writing data visualization code with description of the design choices that led to the implementation (design exposition). We develop a model of design exposition characterised by four visualization designer architypes: the evaluator, the autonomist, the didacticist and the rationalist. The model is used to justify the key characteristics of literate visualization: ‘notebook’ documents that integrate live coding input, rendered output and textual narrative; low cost of authoring textual narrative; guidelines to encourage structured visualization design and its documentation. We propose narrative schemas for structuring and validating a wide range of visualization design approaches and models, and branching narratives for capturing alternative designs and design views. We describe a new open source literate visualization environment, litvis, based on a declarative interface to Vega and Vega-Lite through the functional programming language Elm combined with markdown for formatted narrative. We informally assess the approach, its implementation and potential by considering three examples spanning a range of design abstractions: new visualization idioms; validation though visualization algebra; and feminist data visualization. We argue that the rich documentation of the design process provided by literate visualization offers the potential to improve the validity of visualization design and so benefit both academic visualization and visualization practice.

There is an urgent need to improve the infrastructure supporting the reuse of scholarly data. A diverse set of stakeholders—representing academia, industry, funding agencies, and scholarly publishers—have come together to design and jointly endorse a concise and measureable set of principles that we refer to as the FAIR Data Principles. The intent is that these may act as a guideline for those wishing to enhance the reusability of their data holdings. Distinct from peer initiatives that focus on the human scholar, the FAIR Principles put specific emphasis on enhancing the ability of machines to automatically find and use the data, in addition to supporting its reuse by individuals. This Comment is the first formal publication of the FAIR Principles, and includes the rationale behind them, and some exemplar implementations in the community.

We introduce a system called Penrose for creating mathematical diagrams. Its basic functionality is to translate abstract statements written in familiar math-like notation into one or more possible visual representations. Rather than rely on a fixed library of visualization tools, the visual representation is user-defined in a constraint-based specification language; diagrams are then generated automatically via constrained numerical optimization. The system is user-extensible to many domains of mathematics, and is fast enough for iterative design exploration. In contrast to tools that specify diagrams via direct manipulation or low-level graphics programming, Penrose enables rapid creation and exploration of diagrams that faithfully preserve the underlying mathematical meaning. We demonstrate the effectiveness and generality of the system by showing how it can be used to illustrate a diverse set of concepts from mathematics and computer graphics.

An international association advancing the multidisciplinary study of informing systems. Founded in 1998, the Informing Science Institute (ISI) is a global community of academics shaping the future of informing science.

Background/Context Successful integration of educational technology is a complicated process that is influenced by multiple factors. Recently, both within and across schools, educators have been searching for cloud-based solutions to address the challenges of integrating educational technology into their school systems—assessing whether these programs are affordable, accessible, and well-suited to improve learning. While the popularity of cloud-based applications among educational institutions and students is rapidly increasing due to their enhanced sharing features, accessibility, and cost-efficiency, there have been few efforts to investigate the impacts of these cloud-based applications in educational settings, especially in K-12 settings. Purpose/Objective This paper examines how Google Docs, one of the most popular cloud-based software applications, is integrated into middle-school English Language Arts (ELA) classrooms in a school district with a laptop initiative. Specifically, this case study attempts to understand the contemporary challenges of implementing the collaborative web-based tool and its accompanying opportunities, as well as the contextual factors for its implementation within the district. Research Design This qualitative study followed a grounded approach to data analysis. Using primarily initial coding and thematic coding methods, we analyzed interviews, surveys (from 2,152 students and 25 teachers), classroom observations, and student documents collected over the course of the 2011–2012 academic year. Analysis revealed three key themes (access and workability, cost and practicality, and affordances for writing), as well as the contextual factors of Google Docs implementation (the district's focus on instructional goals and professional development). Conclusion/Recommendations Our case study suggests that the introduction of cloud-based tools was perceived by students, teachers, and district officials to make technology use more accessible and convenient, to enhance cost-efficiency and productivity, and, most importantly, to provide ample affordances for writing practice and instruction. The district-wide implementation of Google Docs provided broad, accessible, and affordable simultaneous access to students and teachers, while increasing their opportunities to improve writing skills through features such as feedback, revision history, and reader selection. We also identified key contextual factors that contributed to these favorable outcomes, such as the district's focus on curricular integration and professional development. As one of the few studies that explores cloud-based tools’ usability and benefits in K-12 settings, we hope to help school districts make informed decisions about adopting these applications for instruction. Though the particularities of context need to be taken into account, the case study nevertheless reveals a cloud-based environment's salient affordances for learning in a district-wide implementation context.

Creating presentation slides is a critical but time-consuming task for data scientists. While researchers have proposed many AI techniques to lift data scientists’ burden on data preparation and model selection, few have targeted the presentation creation task. Based on the needs identified from a formative study, this paper presents NB2Slides, an AI system that facilitates users to compose presentations of their data science work. NB2Slides uses deep learning methods as well as example-based prompts to generate slides from computational notebooks, and take users’ input (e.g., audience background) to structure the slides. NB2Slides also provides an interactive visualization that links the slides with the notebook to help users further edit the slides. A follow-up user evaluation with 12 data scientists shows that participants believed NB2Slides can improve efficiency and reduces the complexity of creating slides. Yet, participants questioned the future of full automation and suggested a human-AI collaboration paradigm.

The aim of the present study was to determine the effect of a collaborative writing programme through the Google Docs writing tool on the production of argumentative discourse within the framework of Modern Greek Language teaching. The participants were 23 pupils in Year 5 who received a six-month period teaching intervation (two hours per week) that included collaborative writing activities of argumentative discourse. Prior to the beginning, and following the end of the programme the performance of all participants on argumentative discourse writing skills was evaluated with a 13-criteria Rubric scale. The results of the paired-sample t-tests showed a statistically significant difference between pre- and post tests in the majority of the criteria and the total scores of the rubric. It is therefore concluded that, the use of a Web platform may positively contribute towards the enhancement of argumentative discourse writing skills of pupils in Year 5 of Primary Education.