Learning without leaving the work
Building a macOS companion for the moments when you know what you want to make, but not what to ask.
Someone opens GarageBand to make music for a video. They have a feeling in mind, but they don’t yet know the language of the tool. They may not know whether they need a loop, a new instrument, or simply a quieter track.
The prototype starts with a smaller question: what are you trying to make? Its GarageBand arc opens by asking what the piece is for, then uses that answer as the test for every loop that follows.
I’m building the companion around screen context, voice, pointing, and an authored GarageBand curriculum. The design question is whether help with this step leaves someone better able to take the next one.
A task, then a handoff
The brief is concrete: make a 16-bar bed for a video with drums, bass, and one melodic layer. The arc separates facts the software might verify—three tracks, about 16 bars, a loop on the timeline, a saved project—from judgments it should leave with the person: groove, balance, and whether the energy suits the video.
Its six steps move from concept to technique to judgment: understand a loop, find one, place it, extend it, then listen back. The machine can help with the structure. It cannot settle whether the music carries the intended feeling.
One task, different kinds of help
“What’s a loop?”
A useful idea before a sequence of instructions.A short piece of sound that repeats. Listen for where it comes back around.
Explain a concept in the context of what they are making.“Where do I find one?”
The idea makes sense. The control is unfamiliar.Open the Loop Browser. Audition a few before choosing one for the timeline.
Connect the instruction to a place in the actual application.“Does this fit my video?”
This is a decision about the work, not the controls.Play it with the video. Does the energy feel right, or does it pull attention away?
Offer a way to judge. Leave the judgment with the person.Concept → technique → judgment. Progress means more than completing the steps.
Help has to leave space
Voice is useful when the eyes and hands are occupied. But a spoken explanation can also compete with the very thing a person needs to hear. In a music tool, silence is part of the interaction.
The curriculum asks before telling, pauses after the first loop is placed, and ends with playback after the software steps away. That is the intended handoff. It is encoded in the arc, but it has not yet been observed in a learner session.
What I’m testing next
The next test is one complete GarageBand task: make the music, work through a real difficulty, then try a related step without help. I want to see where the companion should explain, where it should point, and where it should leave space.
A continuous screen-and-audio recording will let me connect a correction to a specific change in the experience. This remains a working prototype; no learner outcomes are claimed.
A related way to grade claims
The FormFactors evaluator is a separate project, not the scorecard for this companion. It is relevant because it asks how an agent evaluation can avoid flattering itself.
In that separate evaluator, I distinguish an appropriate refusal from a failed attempt and trace corrections back to the action they reversed. A denominator error taught me to test whether each score moves in the right direction. Those rules sharpen what this companion session should record, but they do not supply the missing session.
Sources & current limits
Working-note evidence: the authored GarageBand curriculum and the current companion prototype. A continuous task recording has not yet been produced. No learner outcomes are claimed.
Related thinking: Andy Matuschak on learning through authentic activity. This project is part of an existing conversation about contextual learning, not a claim to have originated it.