Building Real-Time Audio Waveforms with Web Audio API

Creating an interactive audio player that feels native and responsive while handling real-time waveform generation, AWS signed URLs, and track management across a complex React application.

Real-Time Waveform Generation

We built a custom SimpleAudioCanvas component that uses the Web Audio API to generate waveforms in real-time. The component can handle pre-generated waveform data from our storage system, fetch waveforms on-demand, or fall back to demo waveforms for immediate visual feedback. Using AudioContext and decodeAudioData, we process audio files to extract frequency data and create responsive visual representations.

Waveform Generation

// Generate real waveform from audio file
const generateRealWaveform = useCallback(async () => {
    const audioContext = new AudioContext()
    const response = await fetch(audioUrl)
    const arrayBuffer = await response.arrayBuffer()
    const audioBuffer = await audioContext.decodeAudioData(arrayBuffer)
    const channelData = audioBuffer.getChannelData(0)

    // Generate waveform data by sampling
    const waveformArray: number[] = []
    for (let i = 0; i < barCount; i++) {
        const start = i * Math.floor(channelData.length / barCount)
        const end = Math.min(
            start + Math.floor(channelData.length / barCount),
            channelData.length
        )

        // Calculate RMS for this block
        let sum = 0
        for (let j = start; j < end; j++) {
            sum += (channelData[j] || 0) * (channelData[j] || 0)
        }
        const rms = Math.sqrt(sum / (end - start))
        waveformArray.push(Math.min(1, rms * 4))
    }

    setWaveformData(waveformArray)
    setWaveformGenerated(true)
}, [audioUrl, barCount])

AWS Integration & Signed URLs

Our system uses AWS S3 with presigned URLs for secure audio file access. Each track version gets a unique version_key that maps to S3 objects. We implemented a sophisticated caching layer that stores waveform data alongside audio files, reducing processing time and bandwidth. The waveform generation happens during upload, ensuring instant playback without delays.

AWS Integration

// Fetch presigned URL and waveform data
const [presignedUrl, waveformJson] = await Promise.all([
  getStorage().getPresignedUrl({
    key: version.version_key,
    expiresIn: 3600,
  }),
  fetchWaveformJson((version.version_ke·y))
]);

const preGeneratedWaveform = waveformJson?.samples ?? null;

<SimpleAudioCanvas
  audioUrl={presignedAudioUrl}
  preGeneratedWaveform={preGeneratedWaveform}
  trackId={playerId}
  onTrackClick={handleTrackSwitch}
/>

Interactive Canvas with Comments

The audio canvas supports interactive features like click-to-seek, drag-to-scrub, and comment markers. Each comment is positioned based on its timestamp and displays user information in tooltips. The canvas maintains stable positioning even when audio reloads, using refs to prevent comment markers from jumping during playback.

Interactive Canvas

// Handle canvas click for seeking
const handleCanvasClick = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
  const canvas = e.currentTarget;
  const rect = canvas.getBoundingClientRect();
  const progress = Math.max(0, Math.min(1, (e.clientX - rect.left) / rect.width));

  if (isCurrentTrack && duration > 0) {
    seek(progress * duration);
    if (!isPlaying) play();
  } else {
    const targetTime = progress * (trackDuration || duration || 0);
    playTrackFromPosition(trackId, targetTime);
  }
}, [isCurrentTrack, duration, seek, isPlaying, play, trackDuration, playTrackFromPosition, trackId]);

// Comment markers positioning
{commentsWithTimestamps.map((comment) => {
  const timestamp = parseTimestamp(String(comment.audio_timestamp || ''));
  const positionPercent = (timestamp / stableDuration) * 100;

  return (
    <button
      key={comment.id}
      className="absolute bottom-0 -translate-x-1/2 cursor-pointer"
      style={{ left: `${positionPercent}%` }}
      onClick={() => handleCommentClick(timestamp)}
    >
      <span className="size-3.5 rounded-full border border-primary/20 bg-primary/20" />
    </button>
  );
})}

State Management & Performance

We implemented a custom audio store using Zustand that manages global audio state, track switching, and position caching. The system handles multiple audio players simultaneously, with intelligent caching of last positions per track version. Performance optimizations include debounced seeking, efficient canvas redrawing, and proper cleanup of audio contexts.

Audio Store

// Zustand audio store for global state management
interface AudioStore {
    currentTrack: AudioTrack | null
    isPlaying: boolean
    currentTime: number
    duration: number
    lastPositions: Record<string, number>

    play: () => void
    pause: () => void
    seek: (time: number) => void
    playTrackFromPosition: (
        trackId: string,
        startTime?: number
    ) => Promise<void>
    getLastPositionForVersion: (trackId: string, versionId: string) => number
}

// Usage in components
const {
    currentTrack,
    isPlaying,
    currentTime,
    duration,
    seek,
    play,
    playTrackFromPosition,
    getLastPositionForVersion,
} = useAudioStore()

// Get last position for this track version
const lastPosition = getLastPositionForVersion(trackId, versionId)

// Analytics tracking with debounced updates
const sendDurationUpdate = useCallback(
    (isFinalUpdate = false) => {
        if (!playIdRef.current) return

        const totalDurationSeconds = totalListenedTimeRef.current / 1000
        const shouldSendUpdate =
            isFinalUpdate || currentSegmentDuration >= MIN_LISTEN_DURATION_MS

        if (shouldSendUpdate) {
            executeUpdatePlayDuration({
                playId: playIdRef.current,
                durationSeconds: totalDurationSeconds,
            })
        }
    },
    [executeUpdatePlayDuration]
)

The Result

An audio experience that feels like a native desktop app: instant waveform generation, smooth interactions, and error handling that recovers instead of dead-ending. The system scales from demo waveforms to full production audio processing while maintaining consistent performance across different audio formats and file sizes.