--- title: "Binding libVLC directly from Swift 6" description: "An IPTV client needed Picture-in-Picture, and the VLCKit line it ran on does not have it. Notes on owning a C engine rather than consuming one — pointer isolation, lifetimes nothing checks, a patch set carried against VLC's own source, and what that costs." author: "Omar Albeik" date: 2026-08-03 type: article topics: [software, engineering] language: en reading_time_minutes: 21 canonical_url: https://omaralbeik.com/en/blog/binding-libvlc-directly-from-swift-6 translation_url: https://omaralbeik.com/ar/blog/binding-libvlc-directly-from-swift-6 source_url: https://omaralbeik.com/en/blog/binding-libvlc-directly-from-swift-6.md --- # Binding libVLC directly from Swift 6 [Tilfaz](https://tilfaz.app) is an IPTV client. It ships on the App Store, on iPhone, iPad, Mac and Apple TV. IPTV means whatever the provider decided to send: MPEG-TS over UDP, HLS manifests that go stale between segments, MKV containers, SSA subtitles carrying their own styling, audio in codecs Apple has never shipped a decoder for. AVFoundation is very good at the formats Apple ships. "AVFoundation can't do IPTV" is the sort of claim that gets you corrected within the hour by somebody who has shipped a flawless HLS app and never had a reason to feed it anything else. So here is the smallest version of the problem. Take one MP4 carrying H.264 video and AAC audio, and remux it to Matroska with `-c copy`. The elementary streams come out bit-identical — I checked the hashes. The MP4 decodes. The MKV returns `-11828`, *this media format is not supported*. There is no Matroska demuxer, so the question of what is inside is never asked.
The refusals are the cheap half: a loud failure is one you can handle. The expensive rows are the last three. HEVC inside a transport stream opens on macOS, reports `isPlayable == true`, raises nothing, and hands you an asset with no video track — and HEVC in a transport stream is ordinary in IPTV. A viewer does not see an unsupported-format error. They see a channel that plays sound over a black rectangle, and they tell you your app is broken, and they are right. Much later, building demo apps for the library that came out of all this, I needed one file that exercised everything a player ought to cope with. It ended up as sixty seconds of Matroska carrying three video variants, three Opus audio tracks, three subtitle tracks — one of them Arabic, right to left — six named chapters and an attached cover image. Every one of those properties is unreachable through AVFoundation. I built it as a demo. It is really the gap, written down as a file. ## I was already using VLCKit So the engine had to be [libVLC](https://www.videolan.org/vlc/libvlc.html) — VideoLAN's playback core, the same C API that VLC itself runs on. The established Apple wrapper for it is [VLCKit](https://code.videolan.org/videolan/VLCKit), also VideoLAN's own, written in Objective-C. I was already using it. VLCKit is a faithful record of the decade it was written in. I mean that straight: the delegate protocol, the notification names, the `id drawable` you assign a view to — that was how you built a media framework in 2013, and a great deal of software you rely on was built the same way and works. I did not leave over the idiom. I left over Picture-in-Picture, and the reason is duller than a design argument. VLCKit 3 does not have it. There is no PiP type, no PiP protocol, no `VLCDrawable.h` in that line at all — the feature arrives in VLCKit 4, and VLCKit 4 has been in alpha since September 2022 and ships from a download path with the word `unstable` in it. So the actual choice was not modern Swift against good-enough Objective-C. It was: take a production dependency on a four-year-old alpha to get one feature, or bind the C API underneath it myself. Everything downstream of that — the direct C binding, the lifetime rules, most of the patches further down this piece — follows from it.
Two things that table would be dishonest without. The first is that VLCKit 4 is not standing still — it added visionOS and watchOS, rewrote its event handling, gained subsecond seeking, and picked up Swift Package Manager support two weeks before I wrote this. Comparing against the 3.x README, which is what its own repository still shows you, would be comparing against a straw man. The second is that the unreleased engine is mine, not theirs. I am on libVLC 4, which has never had a stable release; VLCKit maintains a line on released libVLC 3 and is still fixing that line. If you need an engine that has shipped, there is exactly one option in this comparison and it is not mine. There is an [open Blocker](https://code.videolan.org/videolan/VLCKit/-/issues/759) on VideoLAN's tracker, filed in July by someone who is not me, describing a second input on a reused player never re-attaching its video output — audio playing behind a permanently black picture, on tvOS, on live MPEG-TS from an IPTV panel. That is channel switching. That is the workload. ## The attempt before the attempt In December 2025 I published [`harflabs/VLC`](https://github.com/harflabs/VLC). Four commits in one evening, and then never again. It was not a libVLC package. It took the three Objective-C frameworks that VLCKit 3.7.0 ships, one per platform, and republished them as SPM binary targets behind a platform-conditional umbrella — VLCKit 3.x has no `Package.swift` at all, and I wanted to type a URL into a manifest like a normal person. It is archived now. The reason is visible in its own manifest, which declares `swift-tools-version: 6.0` and then wraps a framework whose public headers contain not one concurrency annotation. Packaging turned out to be the easy half. I had made VLCKit resolvable and changed nothing about the fact that a Swift 6 SwiftUI app now owned an unannotated, single-delegate object model. [SwiftVLC](https://github.com/harflabs/SwiftVLC) started in February and hit 1.0 in July. The rest of this piece is where those four and a half months went. ## Bind C directly, and accept the bill The first decision was to go from C to Swift with no Objective-C in between. The table above already made that case. What it cannot show is that rendering becomes one view: ```swift struct PlayerView: View { @State private var player = Player() var body: some View { VideoView(player) .onAppear { try? player.play(url: streamURL) } } } ``` `VideoView` hands libVLC an `NSView`/`UIView` through `set_nsobject` and VLC renders into it directly. No layer you construct, no `MTKView`, no `AVPlayerLayer`. The bill is that you now own every C lifetime rule yourself, with no ARC-shaped layer to hide behind. Every libVLC object follows the same pattern — `init` allocates, `deinit` releases, Swift object lifetime owns C pointer lifetime — and that part is mechanical. The interesting failures all happen at the seams. ## Two ways to move a pointer `OpaquePointer` and `UnsafeMutableRawPointer` do not conform to `Sendable`. Not as an oversight and not as a consequence of region-based isolation: the standard library declares the conformance *unavailable*, on purpose, because the compiler cannot reason about what a pointer points at. Regions are what occasionally rescue you by letting a disconnected value transfer between isolation domains. They do not help here, because capturing into an escaping `@Sendable` closure is not a transfer, and releasing a C object off the main thread is exactly that capture. SwiftVLC allows two ways out. For a capture into a single closure, a local binding that opts out: ```swift nonisolated(unsafe) let p = pointer DispatchQueue.global(qos: .utility).async { libvlc_media_player_release(p) } ``` The pointer is trivially transferable and stays valid for the enclosing scope, which is the whole argument. For pointers read and written from several threads over time, a `Mutex` around a state struct, with the struct marked `@unchecked Sendable` and the mutex, not the compiler, doing the actual work. There is a third way, and the architecture document bans it: laundering a pointer through `Int(bitPattern:)` and back. It compiles, it silences the diagnostic, and it destroys the two things that made the diagnostic worth having — the type and the intent. Six months later nobody reading that function can tell a live pointer from an integer, and the compiler has been talked out of helping. The codebase breaks that rule in exactly one place. A qualification path stores a player address as a `UInt` and converts it straight back, and the field's own doc comment offers the banned justification — *stored as bits because `OpaquePointer` is not Sendable*. I have left it there. A rule with one exception you can point at and a rule nobody enforces any more look identical from outside, which is the only reason this paragraph exists. ## Lifetimes nothing checks Events cross three isolation domains: C callbacks firing on libVLC's own threads, a multi-consumer broadcaster, then `@Observable` properties on the main actor.
`Broadcaster.broadcast` snapshots the subscribers under the lock, then runs their filters and yields *outside* it. One line. Two separate things go wrong if you yield inside. A subscriber's filter is user code, and if that code touches the broadcaster again — subscribes, asks whether it is empty, broadcasts — it deadlocks against a mutex that is not recursive; and while it waits, it is still holding a lock libVLC's own event thread is blocked on. Separately, `onTermination` runs synchronously on whatever thread cancelled the consuming task, and it calls `unsubscribe`, which wants that same mutex. A cancellation landing mid-broadcast becomes one thread taking a lock it is already holding. There is a regression test for this now, and it exists because the fix is one line and somebody could move it back. There was not one when the bug was written. Deadlocks of this shape do not turn up in a suite that finishes while you are still watching it; they turn up in the hands of someone scrubbing a live stream on a train. The same type carries two ways to close. `finishAll()` closes the current subscribers and allows resubscribe; `terminate()` closes them and makes every *future* `subscribe` return an immediately-finished stream. The second exists because some broadcasters are reached through a computed property that builds a fresh subscription on each access, so a subscriber arriving after the producer is gone has to get a finished stream rather than a live one nobody will ever feed. Without it, the failure is not a crash. It is an `await` that never returns, which is much worse to find. One callback also has to become two streams, because libVLC mixes two kinds of event on it and only one of them can be dropped safely.
Teardown reads, in prose, like a list of four things in a fixed order. It is not a list. It is a set of constraints that happen to admit that order, and the useful way to hold it is as the question *what is still reading this?*
All three constraints live in one comment above one function, which is the only thing enforcing them. ## A pointer is not an identity This is the idea I would most want to have been told before starting, and it is the one I arrived at last. A `Player` can replace the libVLC player underneath it while the Swift object and its event streams stay alive. When that happens, callbacks already in flight are still going to arrive, and they are going to arrive describing something that no longer exists. The instinct is to compare pointers. The instinct is wrong, and the source says why in one line: *native pointer addresses are not identities — an allocator may reuse a retired address for a later handle.* So every layer that can be replaced under a live consumer gets a monotonic counter, and every event carries the values it was born with.
I did not design this up front. It accreted, one recurring bug class at a time, and when I audited the library after 1.0 the single largest group of findings was some version of *this callback is about the previous thing*. If I were starting again I would reach for it on the first day, and I suspect that is true of anyone binding a C library whose objects outlive the calls that make them. ## Where the wrapper stops being a wrapper The governing fact is that libVLC copies your video-memory callback pointers and their opaque context when a video output *opens*. Clearing the callback variables on the media player afterwards does not revoke a copy that output is already holding. The obvious teardown — unset the callbacks, then free the context — is a use-after-free waiting for a `vout` that has not finished with you yet. The fix ties every retained opaque to one exact `libvlc_media_player_t`, so sequential controllers on the same native handle can hand over atomically while an overlapping output can never touch another output's dimensions, pool, or cleanup state. Retiring an opaque suppresses new display work immediately, but it is released only after the final counted native release for that handle returns and every callback already in flight has drained. No timeout, and no transient `vout` observation, is treated as proof of safety. With C callbacks, "it has probably finished by now" is not a lifetime. Then there are four rendering paths, and they differ mainly in who owns the object at the end.
That last row is the decision I am least comfortable with and most confident about. On macOS, the public sample-buffer mirror crops at 1:1 layer size instead of scaling into the PiP panel, on the macOS releases I support — the video is simply wrong on screen. The path that works is a private framework: load `PIPViewController` out of `PIP.framework` at runtime and reparent VLC's real drawable view into it. None of which means VideoLAN left PiP unsolved. libVLC ships a Picture-in-Picture controller and both wrappers use it. But it is gated to iOS and tvOS in VLC's own build files, the source file says as much in its header, and the class is simply absent from the macOS slice of the binary I ship. There is no native macOS path in the engine to use. So the private-framework path ships behind an explicit opt-in, and every call into a private *Apple* symbol in the library lives in exactly one file. Not because that makes it App Store safe — it does not, and that is the point of the opt-in — but because an auditor asking what private API this touches should be able to read one file and be done. The iOS backend does something similar in miniature: it reports PiP unavailable in the Simulator, deliberately. Simulator AVKit will happily report an active sample-buffer PiP controller while the system window stays black. A test that passes against that is not a passing test, it is a lie with a green tick. And then the abstraction leaks the other way. Getting the geometry I needed out of the vmem path meant adding entry points to libVLC that libVLC does not have — which means shipping an engine whose public C surface I have extended, and then negotiating with archives built before I invented it.
## When you stop reading the headers At some point you stop reading a C engine's headers and start reading its source, and shortly after that you start changing it.
Version 1.0 shipped with six patches. The other nineteen landed twelve days later, in a single eight-day run — which is either convergence or divergence depending on how charitable you feel. I think it is convergence: 1.0 was the point at which I stopped working around the engine and started fixing it. Some are backports of upstream fixes that landed after my pin. The player-timer series is the clearest: on the pinned revision the timer interpolates past the pause point, so a paused player keeps reporting a time that advances. That surfaces directly as `Player.currentTime`, and PiP's control timebase is placed from the same reading, so the visible symptom is a paused video with a drifting scrubber. That patch carries three upstream refactors it depends on, verbatim, rather than hand-adapting the two real fixes onto the older shape — one of the three is what turns the interpolation flag into the enum the real fix adds a case to. A rebase you can re-derive beats a rewrite you have to re-verify. Others are mine. One is a double free in VLC's adaptive demuxer. `ISegment::toChunk()` releases the chunk source twice on the `prepareChunk()` failure path: once explicitly through `recycleSource()`, and again when it deletes the chunk, because `~AbstractChunk()` releases the source the chunk owns. For segment chunks the recycle is not cacheable and goes straight to `delete`, so the second release dispatches through a vtable on freed memory. A one-line commit in 2022 copied a recycle call up from the branch immediately below, where recycling is exactly right: `createChunk()` returned nothing there, so no chunk exists to own the source. One branch up a chunk does exist, and the copy became a second release. Somebody moved a correct line one branch too far. It is still in VLC master today, and there is an [open report](https://code.videolan.org/videolan/vlc/-/work_items/29845) on VideoLAN's tracker describing an intermittent `EXC_BAD_ACCESS` on iOS arm64 after roughly ten minutes of HLS playback, with a stack that lands on the destructor. Ten minutes is not a coincidence. `prepareChunk()` only does work for encrypted segments, and only fails when the key cannot be resolved. Keys are cached. So it bites at key rotation or after cache eviction rather than at startup — which is exactly the *fine for ten minutes, then crash* profile in the report. Encrypted segments only, only when an AES-128 key fails to resolve, flaky network, real device, ten minutes in. That bug is not going to fail your test suite. It did not fail mine, either, and I want to be careful about the order things happened in: somebody else hit it first, on the same engine commit I pin, and filed it upstream. I found the defect by reading the ownership graph, then reproduced it deterministically. My patch adds a regression test to VLC's own adaptive suite; it counts releases instead of freeing, so the double release trips an assertion instead of undefined behaviour. ## I wrote the demos before I wrote the library I built the demo apps first — one per shipping platform, driving every feature I meant to ship, against that sixty-second Matroska — and the test suite alongside them. Both were written as though the library underneath already worked. It did not exist yet. What existed was an executable statement of what working meant: a screen that plays a stream carrying three audio tracks and lets you move between them, a Picture-in-Picture window that survives a seek, a playlist that advances without losing an event on the way. Then the agents closed the gap. They built and ran the apps and the suite, hundreds of times over, on simulators where volume was the point and on real hardware for the things that only fail there. They read the crash logs and opened pull requests. Most of the code in this library came out of that loop, and it is good code, because it was written against a definition of done that already existed and could be checked mechanically. What changed is where my hours went. They stopped going into implementation and started going into specification — into saying what correct meant precisely enough that something other than me could decide whether it had been reached. The issues in that repository are where the work went, and never in the title. It is the acceptance criteria underneath that took the time: ten thousand active media replacements under a sanitizer, with late metadata and teardown callbacks. A seek soak on a physical device, with allocation provenance retained. One combined row covering both PiP backends, so that a passing backend cannot hide a failing one. Writing that well is harder than writing the fix, and it is the part that does not delegate. A loop that runs hundreds of times is worth exactly as much as its ability to say no, and I learned that the expensive way. A batch of tests I had already accepted turned out to contain sixteen that could never reach the condition they were waiting on. They had been passing for months without testing anything. A model will produce a plausible fix and a plausible test for it in the same pass, and neither is worth anything until you break the fix and watch the test fail. So everything else I built exists to make that check cheap. Anything load-bearing carries a red-green note in its commit body, recording that the fix was reverted and the test watched failing. A coverage check reports which patches the binary under test does *not* contain, because CI links a released engine, and a patch in the tree is not a patch in the artifact. And the release gate says what it is: *this does not run the tests — a person does, on hardware. What it enforces is that the results exist, that every required row was executed and passed, and that they describe this artifact rather than an earlier one.* The boundary sat in the same place every time. Volume and patience delegate completely; ownership does not. Lock order, the moment a C object actually dies, what AVKit expects of you that nobody ever wrote down — on those I would get back something reasonable and wrong, and the favourite wrong answer was a timeout where the real answer was a rule about who owns what. A timeout passes. It fails a month later, on someone else's network. Reading VLC's source until the mechanism was understood is what I could not hand over. It is also what produced the patches. Both halves were necessary, and the double free is the proof — in the direction I did not expect. The loop did not find it. Somebody else's crash report did, against the commit I pin, and closing it meant sitting with VLC's source until the vtable dispatch made sense. What the loop bought was everything downstream: the deterministic reproduction, the sanitizer run, the four hundred passes that showed the fix held. Diagnosis was mine. Proof was not. ## What it cost Four and a half months to 1.0, and the bill is still arriving. The patch set is the largest line item, and it is a liability. An ordered patch series against a pinned revision means a rebase every time the pin moves, and pinning does not even buy you a stable world: one of those patches exists solely because a newer Autoconf started handing the C compiler a different standard flag from the one it handed the Objective-C compiler, so libtool could not infer a tag and every Apple slice died in `test/`, after libvlccore and libvlc had both linked. The same sources built a fortnight earlier. The engine ships as a static archive, and that one packaging decision is charged to me twice. Once as a licensing problem: libVLC is LGPL 2.1, static linking pushes a consumer towards the clause that expects you to distribute object files so your users can relink, and an App Store app cannot do that. There is an open issue asking me to ship a dynamic xcframework instead, filed by somebody who does not work on this and who was generous about it — *licensing is the only thing pushing the other way* — and I have not answered it. Dynamic linking would ship a larger binary and I have not decided what that trade is worth. And once as an engineering problem, because a static archive has no identity at load time.
The rest of the ledger, briefly. The macOS PiP path that renders correctly cannot go in the App Store, which means the best version of a feature I built is one I cannot ship. The engine ships stripped of debug symbols and without a dSYM, so a crash inside libVLC gives everyone shipping it a function name and no line number. And physical-device qualification turned into a fifty-three-row matrix and a harness whose own test file is longer than anything I wrote in the library — because the things that break here break on hardware, and nothing else is admissible as proof. If you are considering wrapping a large C library, budget for its source rather than its headers. The API surface looks like the work. It is bounded by the header, and it is the same shape every time. What costs months is the part where the engine's assumptions and your language's guarantees are both correct and incompatible, and only one of them sits in a repository you can edit. I stopped consuming the engine and started owning it, and an engine you own hands you its licensing, its packaging and its bugs together, whether or not you wrote any of them. That is the trade. I would make it again on Monday. But it is a standing obligation, not a milestone you pass. There is no total to give you. Picture-in-Picture on iOS still shuts itself off on a plain seek — open on the tracker as I write this, with a fix merged and not yet proven on hardware, reported by somebody I have never met. It is the feature I left VLCKit to get. Two dozen patches into someone else's engine, and the thing that started all of this is the thing still open.