Binding libVLC directly from Swift 6

Tilfaz is an IPTV client. It ships on the App Store, on iPhone, iPad, Mac and Apple TV. IPTV means whatever the provider decided to send: MPEG-TS over UDP, HLS manifests that go stale between segments, MKV containers, SSA subtitles carrying their own styling, audio in codecs Apple has never shipped a decoder for.

AVFoundation is very good at the formats Apple ships. “AVFoundation can’t do IPTV” is the sort of claim that gets you corrected within the hour by somebody who has shipped a flawless HLS app and never had a reason to feed it anything else. So here is the smallest version of the problem.

Take one MP4 carrying H.264 video and AAC audio, and remux it to Matroska with -c copy. The elementary streams come out bit-identical — I checked the hashes. The MP4 decodes. The MKV returns -11828, this media format is not supported. There is no Matroska demuxer, so the question of what is inside is never asked.

• decoded
• opens, tracks missing
• refused — −11828

fixture	macOS 26	iOS 26
base.mp4H.264 · AAC
h264_aac.mkvsame streams, remuxed
vp9_opus.webmVP9 · Opus
h264_aac.flvH.264 · AAC
h264_ac3.mp4Dolby Digital
h264_aac.tsH.264 · AAC
h264_mp2.tsMPEG-1 Layer II
hevc_aac.tsHEVC · AAC	no video
h264_dts.tsH.264 · DTS	no audio
h264_opus.tsH.264 · Opus	no audio

Every partial success reports isPlayable == true and raises nothing. HEVC in a transport stream is ordinary in IPTV.

The refusals are the cheap half: a loud failure is one you can handle. The expensive rows are the last three. HEVC inside a transport stream opens on macOS, reports isPlayable == true, raises nothing, and hands you an asset with no video track — and HEVC in a transport stream is ordinary in IPTV. A viewer does not see an unsupported-format error. They see a channel that plays sound over a black rectangle, and they tell you your app is broken, and they are right.

Much later, building demo apps for the library that came out of all this, I needed one file that exercised everything a player ought to cope with. It ended up as sixty seconds of Matroska carrying three video variants, three Opus audio tracks, three subtitle tracks — one of them Arabic, right to left — six named chapters and an attached cover image. Every one of those properties is unreachable through AVFoundation. I built it as a demo. It is really the gap, written down as a file.

I was already using VLCKit

So the engine had to be libVLC — VideoLAN’s playback core, the same C API that VLC itself runs on. The established Apple wrapper for it is VLCKit, also VideoLAN’s own, written in Objective-C. I was already using it.

VLCKit is a faithful record of the decade it was written in. I mean that straight: the delegate protocol, the notification names, the id drawable you assign a view to — that was how you built a media framework in 2013, and a great deal of software you rely on was built the same way and works.

I did not leave over the idiom. I left over Picture-in-Picture, and the reason is duller than a design argument. VLCKit 3 does not have it. There is no PiP type, no PiP protocol, no VLCDrawable.h in that line at all — the feature arrives in VLCKit 4, and VLCKit 4 has been in alpha since September 2022 and ships from a download path with the word unstable in it. So the actual choice was not modern Swift against good-enough Objective-C. It was: take a production dependency on a four-year-old alpha to get one feature, or bind the C API underneath it myself. Everything downstream of that — the direct C binding, the lifetime rules, most of the patches further down this piece — follows from it.

Where the wrapper is ahead

SwiftVLCVLCKit 4.0.0-a22

Call path

Swift 6, C bound directly

Objective-C — no Swift in the sources

Concurrency

Main-actor player, Sendable media

None. Callbacks land on libVLC's thread

Events

An AsyncStream each, buffered per lane

One delegate, plus NotificationCenter

Errors

Typed throws, nine cases

No error type — nil, BOOL, a thread-local string

SwiftUI

A view ships

Write your own representable

Where VLCKit is ahead

SwiftVLCVLCKit 4.0.0-a22

Linkage

A static archive in every slice

A dynamic framework — static dropped in 4.0

LGPL relink route

None that an App Store app can take

The conventional one, and VideoLAN's own

Crash reports

Stripped — libVLC symbolicates to nothing

Debug symbols ship inside the framework

Reach

iOS 18 up. Catalyst, which VLCKit lacks

iOS 12, macOS 10.13, watchOS

Engine

libVLC 4 — never released stable

Also a line on released libVLC 3

Track record

One app

VLC for iOS, on this alpha, today

Hands on it

One

VideoLAN, on sources dating to 2007

Both carry their own patch set against VLC — VLCKit 27, SwiftVLC 25. Patching the engine is the ordinary cost of shipping libVLC on Apple platforms, not a distinguishing feature of either.

Two things that table would be dishonest without. The first is that VLCKit 4 is not standing still — it added visionOS and watchOS, rewrote its event handling, gained subsecond seeking, and picked up Swift Package Manager support two weeks before I wrote this. Comparing against the 3.x README, which is what its own repository still shows you, would be comparing against a straw man.

The second is that the unreleased engine is mine, not theirs. I am on libVLC 4, which has never had a stable release; VLCKit maintains a line on released libVLC 3 and is still fixing that line. If you need an engine that has shipped, there is exactly one option in this comparison and it is not mine.

There is an open Blocker on VideoLAN’s tracker, filed in July by someone who is not me, describing a second input on a reused player never re-attaching its video output — audio playing behind a permanently black picture, on tvOS, on live MPEG-TS from an IPTV panel. That is channel switching. That is the workload.

The attempt before the attempt

In December 2025 I published harflabs/VLC. Four commits in one evening, and then never again.

It was not a libVLC package. It took the three Objective-C frameworks that VLCKit 3.7.0 ships, one per platform, and republished them as SPM binary targets behind a platform-conditional umbrella — VLCKit 3.x has no Package.swift at all, and I wanted to type a URL into a manifest like a normal person.

It is archived now. The reason is visible in its own manifest, which declares swift-tools-version: 6.0 and then wraps a framework whose public headers contain not one concurrency annotation. Packaging turned out to be the easy half. I had made VLCKit resolvable and changed nothing about the fact that a Swift 6 SwiftUI app now owned an unannotated, single-delegate object model.

SwiftVLC started in February and hit 1.0 in July. The rest of this piece is where those four and a half months went.

Bind C directly, and accept the bill

The first decision was to go from C to Swift with no Objective-C in between.

The table above already made that case. What it cannot show is that rendering becomes one view:

struct PlayerView: View {
  @State private var player = Player()

  var body: some View {
    VideoView(player)
      .onAppear { try? player.play(url: streamURL) }
  }
}

VideoView hands libVLC an NSView/UIView through set_nsobject and VLC renders into it directly. No layer you construct, no MTKView, no AVPlayerLayer.

The bill is that you now own every C lifetime rule yourself, with no ARC-shaped layer to hide behind. Every libVLC object follows the same pattern — init allocates, deinit releases, Swift object lifetime owns C pointer lifetime — and that part is mechanical. The interesting failures all happen at the seams.

Two ways to move a pointer

OpaquePointer and UnsafeMutableRawPointer do not conform to Sendable. Not as an oversight and not as a consequence of region-based isolation: the standard library declares the conformance unavailable, on purpose, because the compiler cannot reason about what a pointer points at. Regions are what occasionally rescue you by letting a disconnected value transfer between isolation domains. They do not help here, because capturing into an escaping @Sendable closure is not a transfer, and releasing a C object off the main thread is exactly that capture.

SwiftVLC allows two ways out. For a capture into a single closure, a local binding that opts out:

nonisolated(unsafe) let p = pointer
DispatchQueue.global(qos: .utility).async {
  libvlc_media_player_release(p)
}

The pointer is trivially transferable and stays valid for the enclosing scope, which is the whole argument. For pointers read and written from several threads over time, a Mutex around a state struct, with the struct marked @unchecked Sendable and the mutex, not the compiler, doing the actual work.

There is a third way, and the architecture document bans it: laundering a pointer through Int(bitPattern:) and back. It compiles, it silences the diagnostic, and it destroys the two things that made the diagnostic worth having — the type and the intent. Six months later nobody reading that function can tell a live pointer from an integer, and the compiler has been talked out of helping.

The codebase breaks that rule in exactly one place. A qualification path stores a player address as a UInt and converts it straight back, and the field’s own doc comment offers the banned justification — stored as bits because OpaquePointer is not Sendable. I have left it there. A rule with one exception you can point at and a rule nobody enforces any more look identical from outside, which is the only reason this paragraph exists.

Lifetimes nothing checks

Events cross three isolation domains: C callbacks firing on libVLC’s own threads, a multi-consumer broadcaster, then @Observable properties on the main actor.

Broadcaster.broadcast snapshots the subscribers under the lock, then runs their filters and yields outside it. One line.

Two separate things go wrong if you yield inside. A subscriber’s filter is user code, and if that code touches the broadcaster again — subscribes, asks whether it is empty, broadcasts — it deadlocks against a mutex that is not recursive; and while it waits, it is still holding a lock libVLC’s own event thread is blocked on. Separately, onTermination runs synchronously on whatever thread cancelled the consuming task, and it calls unsubscribe, which wants that same mutex. A cancellation landing mid-broadcast becomes one thread taking a lock it is already holding.

There is a regression test for this now, and it exists because the fix is one line and somebody could move it back. There was not one when the bug was written. Deadlocks of this shape do not turn up in a suite that finishes while you are still watching it; they turn up in the hands of someone scrubbing a live stream on a train.

The same type carries two ways to close. finishAll() closes the current subscribers and allows resubscribe; terminate() closes them and makes every future subscribe return an immediately-finished stream. The second exists because some broadcasters are reached through a computed property that builds a fresh subscription on each access, so a subscriber arriving after the producer is gone has to get a finished stream rather than a live one nobody will ever feed. Without it, the failure is not a crash. It is an await that never returns, which is much worse to find.

One callback also has to become two streams, because libVLC mixes two kinds of event on it and only one of them can be dropped safely.

one C callback, on libVLC's thread

control24 cases

Media identity, capability, lifecycle, terminal outcomes.

Each reports something that happened once. Nothing later re-states it, so a dropped one is lost information.

unboundedMemory grows with consumer lag times the control rate, not the clock rate.

timing4 cases

The playback clock, buffer fill, render counters.

Each supersedes the one before it, so dropping a backlog costs resolution and nothing else.

newest 4A lagging consumer skips stale samples and still receives the newest.

timeChanged alone fires around 30 Hz. Through one bounded buffer, a consumer stalled for two seconds loses whatever was queued behind it — a mediaChanged, an endReached.

The limit it does not solve

Newest-4 is across the timing lane, not one slot per kind, so a long enough burst of the fastest kind can still evict the newest of a slower one. What the split does guarantee is the half that matters: nothing dropped can ever be a one-shot control event, because control events are not in that buffer.

Teardown reads, in prose, like a list of four things in a fixed order. It is not a list. It is a set of constraints that happen to admit that order, and the useful way to hold it is as the question what is still reading this?

All three constraints live in one comment above one function, which is the only thing enforcing them.

A pointer is not an identity

This is the idea I would most want to have been told before starting, and it is the one I arrived at last.

A Player can replace the libVLC player underneath it while the Swift object and its event streams stay alive. When that happens, callbacks already in flight are still going to arrive, and they are going to arrive describing something that no longer exists. The instinct is to compare pointers. The instinct is wrong, and the source says why in one line: native pointer addresses are not identities — an allocator may reuse a retired address for a later handle.

So every layer that can be replaced under a live consumer gets a monotonic counter, and every event carries the values it was born with.

I did not design this up front. It accreted, one recurring bug class at a time, and when I audited the library after 1.0 the single largest group of findings was some version of this callback is about the previous thing. If I were starting again I would reach for it on the first day, and I suspect that is true of anyone binding a C library whose objects outlive the calls that make them.

Where the wrapper stops being a wrapper

The governing fact is that libVLC copies your video-memory callback pointers and their opaque context when a video output opens. Clearing the callback variables on the media player afterwards does not revoke a copy that output is already holding. The obvious teardown — unset the callbacks, then free the context — is a use-after-free waiting for a vout that has not finished with you yet.

The fix ties every retained opaque to one exact libvlc_media_player_t, so sequential controllers on the same native handle can hand over atomically while an overlapping output can never touch another output’s dimensions, pool, or cleanup state. Retiring an opaque suppresses new display work immediately, but it is released only after the final counted native release for that handle returns and every callback already in flight has drained. No timeout, and no transient vout observation, is treated as proof of safety. With C callbacks, “it has probably finished by now” is not a lifetime.

Then there are four rendering paths, and they differ mainly in who owns the object at the end.

• VideoViewall platforms

  set_nsobjectNSView / UIView

  VLC draws into your view

• PiPVideoViewiOS

  drawable proxyVLC's iOS sample-buffer voutAVPictureInPictureController

  controller owned by libVLC

• PiPControllerdirect, public API

  vmem callbacksCVPixelBufferCMSampleBufferAVSampleBufferDisplayLayer

  layer owned by SwiftVLC · 8-bit BGRA, SDR

• PiPVideoViewmacOS, SPI opt-in

  set_nsobjectVLC's own NSViewPIPViewController

  private PIP.framework · off by default

That last row is the decision I am least comfortable with and most confident about. On macOS, the public sample-buffer mirror crops at 1:1 layer size instead of scaling into the PiP panel, on the macOS releases I support — the video is simply wrong on screen. The path that works is a private framework: load PIPViewController out of PIP.framework at runtime and reparent VLC’s real drawable view into it.

None of which means VideoLAN left PiP unsolved. libVLC ships a Picture-in-Picture controller and both wrappers use it. But it is gated to iOS and tvOS in VLC’s own build files, the source file says as much in its header, and the class is simply absent from the macOS slice of the binary I ship. There is no native macOS path in the engine to use.

So the private-framework path ships behind an explicit opt-in, and every call into a private Apple symbol in the library lives in exactly one file. Not because that makes it App Store safe — it does not, and that is the point of the opt-in — but because an auditor asking what private API this touches should be able to read one file and be done.

The iOS backend does something similar in miniature: it reports PiP unavailable in the Simulator, deliberately. Simulator AVKit will happily report an active sample-buffer PiP controller while the system window stays black. A test that passes against that is not a passing test, it is a lie with a green tick.

And then the abstraction leaks the other way. Getting the geometry I needed out of the vmem path meant adding entry points to libVLC that libVLC does not have — which means shipping an engine whose public C surface I have extended, and then negotiating with archives built before I invented it.

the handshake

swiftvlc_libvlc_pip_extensions_version()

returns 0

a released archive that predates the symbols

weak definitions in the Swift shim — they exist only so the link succeeds

fall back to the public format callbacks

crop and pixel aspect cannot be proven

returns 3

an archive built from the patched tree

the engine's own strong definitions win

the extended format callback, with geometry captured atomically

coded size, visible size, crop and aspect arrive together or not at all

What each revision of the extension added

1. 1geometry-aware vmem setup, and a media-and-length snapshot taken under the player lock
2. 2a playback snapshot as a distinct type, so a newer archive cannot write past storage a older client allocated
3. 3overlay composition — version-gated only, with no weak stub to fall back to

A header installed into VLC's own include list, exported symbols in its symbol file, and a static assertion pinning the struct's size and every field offset. The wrapper now has an ABI contract with itself.

When you stop reading the headers

At some point you stop reading a C engine’s headers and start reading its source, and shortly after that you start changing it.

When they arrived

619

• shipped with 1.0
• written in the nine days after it

What they touch

• Apple video output & PiP8
• libVLC C API8
• Input & player core6
• Demuxers — MP4, TS, HLS4
• Build & test scaffolding3
• Chromecast & stream out2
• UPnP discovery2
• avcodec1

0246810

patches touching this area

Where they came from

9reproduce upstream commits

40 distinct ones, named in the patch headers — two of the nine disclose adaptations to the pin

16written here

original fixes, new C API surface, and build repairs upstream has no reason to carry

What they land on

56

VLC files modified

6

files createda public header, three sample-buffer geometry headers, and two regression tests added to VLC's own suites

Version 1.0 shipped with six patches. The other nineteen landed twelve days later, in a single eight-day run — which is either convergence or divergence depending on how charitable you feel. I think it is convergence: 1.0 was the point at which I stopped working around the engine and started fixing it.

Some are backports of upstream fixes that landed after my pin. The player-timer series is the clearest: on the pinned revision the timer interpolates past the pause point, so a paused player keeps reporting a time that advances. That surfaces directly as Player.currentTime, and PiP’s control timebase is placed from the same reading, so the visible symptom is a paused video with a drifting scrubber. That patch carries three upstream refactors it depends on, verbatim, rather than hand-adapting the two real fixes onto the older shape — one of the three is what turns the interpolation flag into the enum the real fix adds a case to. A rebase you can re-derive beats a rewrite you have to re-verify.

Others are mine. One is a double free in VLC’s adaptive demuxer.

ISegment::toChunk() releases the chunk source twice on the prepareChunk() failure path: once explicitly through recycleSource(), and again when it deletes the chunk, because ~AbstractChunk() releases the source the chunk owns. For segment chunks the recycle is not cacheable and goes straight to delete, so the second release dispatches through a vtable on freed memory.

A one-line commit in 2022 copied a recycle call up from the branch immediately below, where recycling is exactly right: createChunk() returned nothing there, so no chunk exists to own the source. One branch up a chunk does exist, and the copy became a second release. Somebody moved a correct line one branch too far. It is still in VLC master today, and there is an open report on VideoLAN’s tracker describing an intermittent EXC_BAD_ACCESS on iOS arm64 after roughly ten minutes of HLS playback, with a stack that lands on the destructor.

Ten minutes is not a coincidence. prepareChunk() only does work for encrypted segments, and only fails when the key cannot be resolved. Keys are cached. So it bites at key rotation or after cache eviction rather than at startup — which is exactly the fine for ten minutes, then crash profile in the report.

Encrypted segments only, only when an AES-128 key fails to resolve, flaky network, real device, ten minutes in. That bug is not going to fail your test suite. It did not fail mine, either, and I want to be careful about the order things happened in: somebody else hit it first, on the same engine commit I pin, and filed it upstream. I found the defect by reading the ownership graph, then reproduced it deterministically. My patch adds a regression test to VLC’s own adaptive suite; it counts releases instead of freeing, so the double release trips an assertion instead of undefined behaviour.

I wrote the demos before I wrote the library

I built the demo apps first — one per shipping platform, driving every feature I meant to ship, against that sixty-second Matroska — and the test suite alongside them. Both were written as though the library underneath already worked. It did not exist yet. What existed was an executable statement of what working meant: a screen that plays a stream carrying three audio tracks and lets you move between them, a Picture-in-Picture window that survives a seek, a playlist that advances without losing an event on the way.

Then the agents closed the gap. They built and ran the apps and the suite, hundreds of times over, on simulators where volume was the point and on real hardware for the things that only fail there. They read the crash logs and opened pull requests. Most of the code in this library came out of that loop, and it is good code, because it was written against a definition of done that already existed and could be checked mechanically.

What changed is where my hours went. They stopped going into implementation and started going into specification — into saying what correct meant precisely enough that something other than me could decide whether it had been reached. The issues in that repository are where the work went, and never in the title. It is the acceptance criteria underneath that took the time: ten thousand active media replacements under a sanitizer, with late metadata and teardown callbacks. A seek soak on a physical device, with allocation provenance retained. One combined row covering both PiP backends, so that a passing backend cannot hide a failing one. Writing that well is harder than writing the fix, and it is the part that does not delegate.

A loop that runs hundreds of times is worth exactly as much as its ability to say no, and I learned that the expensive way. A batch of tests I had already accepted turned out to contain sixteen that could never reach the condition they were waiting on. They had been passing for months without testing anything. A model will produce a plausible fix and a plausible test for it in the same pass, and neither is worth anything until you break the fix and watch the test fail.

So everything else I built exists to make that check cheap. Anything load-bearing carries a red-green note in its commit body, recording that the fix was reverted and the test watched failing. A coverage check reports which patches the binary under test does not contain, because CI links a released engine, and a patch in the tree is not a patch in the artifact. And the release gate says what it is: this does not run the tests — a person does, on hardware. What it enforces is that the results exist, that every required row was executed and passed, and that they describe this artifact rather than an earlier one.

The boundary sat in the same place every time. Volume and patience delegate completely; ownership does not. Lock order, the moment a C object actually dies, what AVKit expects of you that nobody ever wrote down — on those I would get back something reasonable and wrong, and the favourite wrong answer was a timeout where the real answer was a rule about who owns what. A timeout passes. It fails a month later, on someone else’s network. Reading VLC’s source until the mechanism was understood is what I could not hand over. It is also what produced the patches.

Both halves were necessary, and the double free is the proof — in the direction I did not expect. The loop did not find it. Somebody else’s crash report did, against the commit I pin, and closing it meant sitting with VLC’s source until the vtable dispatch made sense. What the loop bought was everything downstream: the deterministic reproduction, the sanitizer run, the four hundred passes that showed the fix held. Diagnosis was mine. Proof was not.

What it cost

Four and a half months to 1.0, and the bill is still arriving.

The patch set is the largest line item, and it is a liability. An ordered patch series against a pinned revision means a rebase every time the pin moves, and pinning does not even buy you a stable world: one of those patches exists solely because a newer Autoconf started handing the C compiler a different standard flag from the one it handed the Objective-C compiler, so libtool could not infer a tag and every Apple slice died in test/, after libvlccore and libvlc had both linked. The same sources built a fortnight earlier.

The engine ships as a static archive, and that one packaging decision is charged to me twice. Once as a licensing problem: libVLC is LGPL 2.1, static linking pushes a consumer towards the clause that expects you to distribute object files so your users can relink, and an App Store app cannot do that. There is an open issue asking me to ship a dynamic xcframework instead, filed by somebody who does not work on this and who was generous about it — licensing is the only thing pushing the other way — and I have not answered it. Dynamic linking would ship a larger binary and I have not decided what that trade is worth.

And once as an engineering problem, because a static archive has no identity at load time.

why it mattersThe engine ships as a static archive, and a static archive has no identity at load time. Two images that each link it produce two complete libVLC runtimes in one process — two plugin registries, and fifteen unnamespaced Objective-C classes defined twice over, one of which is called simply AoutWrapper. Nothing fails to link. The runtime picks one of each and does not say which.

the supported shape

• App
• FeatureAstatic
• FeatureBstatic
• MediaCoredynamic
• SwiftVLC
• libvlc.a

what CI counts

_libvlc_new

loaded image	defines it
App executable	0
FeatureA	0
FeatureB	0
MediaCore.framework	1

Exactly one, and it has to be that one.

The rest of the ledger, briefly. The macOS PiP path that renders correctly cannot go in the App Store, which means the best version of a feature I built is one I cannot ship. The engine ships stripped of debug symbols and without a dSYM, so a crash inside libVLC gives everyone shipping it a function name and no line number. And physical-device qualification turned into a fifty-three-row matrix and a harness whose own test file is longer than anything I wrote in the library — because the things that break here break on hardware, and nothing else is admissible as proof.

If you are considering wrapping a large C library, budget for its source rather than its headers. The API surface looks like the work. It is bounded by the header, and it is the same shape every time. What costs months is the part where the engine’s assumptions and your language’s guarantees are both correct and incompatible, and only one of them sits in a repository you can edit.

I stopped consuming the engine and started owning it, and an engine you own hands you its licensing, its packaging and its bugs together, whether or not you wrote any of them. That is the trade. I would make it again on Monday. But it is a standing obligation, not a milestone you pass.

There is no total to give you. Picture-in-Picture on iOS still shuts itself off on a plain seek — open on the tracker as I write this, with a fix merged and not yet proven on hardware, reported by somebody I have never met. It is the feature I left VLCKit to get. Two dozen patches into someone else’s engine, and the thing that started all of this is the thing still open.