- In eBPF-land you're going to be calling C functions in the kernel, and using (generally) C data types like structs and null-terminated strings. You can't do loops (loops are unrolled by the compiler), you can't do variadic functions, and you definitely can't take advantage of all the cool Go stuff like goroutines, select, context, etc.
I'm not really sure why you'd want to use this. If you're writing eBPF, you already need to know how to read C kernel source.
- Nitpick: you definitely can do loops as long as the verifier can prove they're bounded.
At my previous job, I've written production eBPF exclusively in Rust using Aya (mentioned by a sibling comment), and it's been a blast. Being able to share the type definitions between the kernel-space and the user-space code is a blessing to avoid subtle issues when going through the maps. And, at least in Rust, you can re-use crates and types that make you gain time. As a (simple) example, being able to use the standard library's IpAddr types or the ipnet crate to not have to roll your own IP and network manipulation libraries is a (small) timesave. It's main value is not needing to onboard new developers.
The Rust type system is a good helper in keeping the verifier happy. Slices, iterators, match statements, etc are very good in my experience (e.g. Option is a godsend to ensure you stay withing the bounds of the input packet, esp. slice::split_at when parsing headers).
But you're right that reading C is non-negotiatable, especially since pretty much all example code on the internet is in C.
- It's the same situation with Aya in Rust. It's sort of a "same language on the backend and the frontend" kind of deal, like with Javascript.
I agree though, I think it makes more sense just to write the C code. The hard part of eBPF isn't writing the code; it's getting it past the verifier.
- > Why transpile, not generate BPF directly
> gc, the Go compiler, has no LLVM-based BPF backend. Adding one is a multi-year compiler project. rustc is built on LLVM and that's why Aya works. So gobee emits C and reuses clang's BPF backend, which gives us mature codegen, BTF, and CO-RE relocations for free.
I wonder if TinyGo (https://tinygo.org/) might be a better fit here:
> TinyGo brings the Go programming language to embedded systems and to the modern web by creating a new compiler based on LLVM.
I've not played with TinyGo much so would be interested to hear other peoples experiences.
- Here's another option.. I created an optimizing eBPF compiler in Common Lisp for a lisp-ish DSL. It's nice because you can compile and load your eBPF code all in-process in lisp (even from your REPL) without any external tooling. https://github.com/atgreen/Whistler
- this is the way
- Well tinygo takes some go bindings they implemented for llvm, https://github.com/tinygo-org/go-llvm, uses the Go standard library for parsing, and wires it up to a LLVM IR generator, with a set of flexible backend/machine definition machinery.
You could likely improve gobee to use tinygo's packages directly, instead of transpiling to C and calling into clang, and the licenses of the two projects look compatible. You'll still need to deal with defining a subset to pass the verifier, of course.
---
From the README:
> Replace clang. clang's BPF backend gives us CO-RE, BTF, and verifier-friendly codegen for free. Reimplementing that costs years and gains nothing.
The primary gotcha you may hit if you try this is how much of the BPF features are implemented by clang, and how much is instead implemented in core LLVM. Even with a LLVM sitting next door you could pull out, the harnesses may not exist independent of clang, but I have not looked THAT deep.
- I did work on a plugin system for Filestash leveraging wasm. Plugins made with tinygo were 10x slower than the same code in rust or c compiled to wasm
[1] https://github.com/mickael-kerjean/filestash/blob/master/ser...
- Yeah I’m not surprised by that. TinyGo might leverage LLVM but it still has all of the Go conveniences that would make it benchmark slower compared to C and Rust. Though I’m sure that gap could be closer if TinyGo had the same level of investment that Rust or Googles Go compiler had.
What I was more curious about was how TinyGo compared with Googles Go compiler and whether TinyGo’s LLVM compiler is compatible with vanilla LLVM compilers (eg can TinyGo compile to all the same targets that, for example Rust, could?)
- We write quite a bit of BPF at work for the Parca-Agent project[1]. And we find that even C is sometimes too high level of a language paired with modern optimization techniques as the code you write often doesn’t come out that way the other and and we have to resort to weird hacks or write assembly directly to appease the verifier.
Apart from that, the usual qualms one might have about C are not really relevant in eBPF land, so I’ve actually found it the nicest experience writing C I’ve ever had, the verifier is just the price we have to pay.
- Yeah I actually advocate for dropping to assembly quite a lot in BPF:
- portability isn't a concern
- BPF ASM syntax is quite readable
- it can often let you write simpler code by directly doing what the verifier needs instead of dancing around trying to make Clang do it for you.
I think the most exciting alternative BPF language would be one where the compiler interacts with the verifier. E.g. if the program included a logical proof of correctness that the verifier could check more efficiently than its limited builtin analysis.
- I'm primarily a Go developer and love the language and will defend it for most use-cases, but to be honest BPF seems like Rust's place to shine.
- I think the real benefit here is being able to share structures, etc. with userspace and keep them in sync.
If this was compiling the Golang to BPF then yeah, that would feel ridiculous, but given that it's transpiling instead then, assuming that it's generating correct and reasonable code, I think this is certainly fine enough. Especially if you're just writing a proof of concept or something pretty basic, there's no reason not to start here.
If you're doing something like trying to filter 40Gbps of network traffic in eBPF then you'd probably want to consider something more hand-tuned/low-level, but that might well be a premature optimization for all I know.
- I feel like every language has its fans. What invariably happens is that people want their favorite language to work in every situation that they might need to work.
Personally I would choose Rust as well, but I would choose Rust for almost everything I do. I can see why a Go developer would want a similar experience.
- I'm also primarily a Go developer, and I will also defend the language in almost all use cases, but I personally feel that C is the best for writing eBPF. I just feel that you get all of the original functionality with C, and you don't have to hack your way around weird issues that you would encounter when using Go or Rust.
Note that we at Bomfather have our userspace code written in Golang and our eBPF code written in C.
But, either way, this is a really cool solution/idea and could make writing eBPF code a lot easier.
- Rust is in the same boat, ebpf is C.
- Why? What value does Rust add here?
- spoken like a true go developer ha
- Johannes Bechberge mades a blog post series about writing ebpf in pure Java : https://mostlynerdless.de/?s=ebpf&submit=Search Also several talks on youtube about this.
- Remember, a lot of the memory safety benefits from go and rust and eBPF don't apply to the kernel eBPF! Kernel eBPF enforces semantics that verify array and loop bounds, memory accesses, and correctness of programs via the verifier. I think for most usecases, it is still best to write eBPF in C!
- Noob question: why did they not choose to use WebAssembly in the kernel instead?
- eBPF has a lot more checks in the verifier, which you could read about here if you're interested in learning more: https://docs.kernel.org/bpf/verifier.html
Since you don't want to handle any kind of crash, out of bounds exception, etc., the eBPF verifier does a ton of impressively paranoid stuff. It ensures that the program doesn't loop (or if it loops that the loop is provably bounded and cannot be infinite), it guarantees that you don't read from a register that might not have been written to, etc.
Basically, it needs to be able to mathematically prove without a doubt that the program behaves as it's supposed to or the verifier refuses to load it at all. WASM doesn't do that, since WASM is a general-purpose 'machine' and WASM programs could theoretically just run forever in entirely reasonable cases.
- eBPF predates WebAssembly by a few years. I'm also not sure Linux would've wanted to integrate and rely so heavily on a standard they aren't in control of the design of.
- eBPF has much stronger constraints than WebAssembly.
- if you crash in wasm, your tab dies. if you crash in ebpf, your server dies. different stakes.
- Lately I’ve been using Go for a personal project and I am so so happy about it. So so happy.
- I hate writing in Golang, I really do, but I cannot deny that it does what it set out to do extremely well.
The tooling is phenomenal and fast. It won't let me accidentally not use a variable, meaning that it won't let me foo, err := something() and not check err. It makes a lot of stuff explicit (e.g. there's no `array.add(item)`, just `array = append(array, newitem)` which makes it more obvious that I might be creating a lot more arrays than just the one I'm trying to work with, but it lets me do `make([]string, 5000)` to pre-allocate the length I want if I know what I need.
Every variable type has a default 'empty' value that is a valid value; an int with no value assigned is 0, a string with no value assigned is "", so you never get corrupted or random data when one of your branches doesn't set the value.
It has a lot of nice thread-safety stuff, since goroutines are such a thing. There's built-in functionality to say "Spawn all these goroutines and then wait until they're done", but there's also functionality to say "Here's a function, it should be called at most once across the lifetime of the program" so that you don't have to manually synchronize "did I do this initialization yet? Is it done yet? Get a lock and then check everything and then set everything."
And it's fast. It's really, really fast. It's so fast that I was testing a GOCACHEPROG program to cache intermediate compilation results instead of recompiling them and in at least some cases it was faster to recompile than to use the cache. The cache was a cloud storage bucket in another country, mind you, but with Rust or C++ that would still be a huge win. With Golang I had to work really, really hard to get cloud storage of intermediate artifacts to be faster than just recompiling on my laptop.
So yeah, I hate Golang and I hate writing Golang but... yeah, it's pretty good.
- Fun fact: naming identifiers with leading underscores in C conflicts with reserved use and should always be avoided. I noticed Gobee declares double-underscores liberally.
Per 6.4.3 (Identifiers) of C23 (ISO/IEC 9899:202y N3886):
https://open-std.org/JTC1/SC22/WG14/www/docs/n3886.pdf— All identifiers that begin with a double underscore (__) or begin with an underscore (_) followed by an uppercase letter are reserved for any use, except those identifiers which are lexically identical to keywords. — All identifiers that begin with an underscore are reserved for use as identifiers with file scope in both the ordinary and tag name spaces.And per 7.1.3 (Reserved identifiers) of C11 (ISO/IEC 9899:201x N1570):
https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf— All identifiers that begin with an underscore and either an uppercase letter or another underscore are always reserved for any use. - Clean implementation. One thing I always look for: how does this degrade when things go wrong? Good error handling is what separates weekend projects from tools people actually use.
- Sorry, why wouldn’t I write in the native language?
- This transpiles to the native language.
- What is BPF?
- I'm guessing Berkeley Packet Filter: https://en.wikipedia.org/wiki/Berkeley_Packet_Filter
This is why National Aeronautics & Space Administration (NASA) guidance is the following:
> Acronyms often confuse readers. Avoid them whenever possible. If an acronym is necessary for future reference, spell the full word and follow with the acronym in parentheses on the first reference. For example, The General Services Administration (GSA).
https://nasa.github.io/content-guide/abbreviations-and-acron...
There is also this longer memo on the NASA Technical Reports Server: https://ntrs.nasa.gov/citations/19950025292
- In fairness to the authors README, there isn't really any other BPFs in this domain and if someone didn't already know what BPF was, then this project wouldn't be of any interest to them (and would be a bad place to start on an BPF journey too).
So I can't blame the project's author for the lack of explanation about what BPF is. Particularly when it's just someone's personal project.
And before anyone complains about this comment: I do think the GP is completely fair in asking for clarification as to what BPF is too. There sometimes seems to be backlash on HN against people asking for a term to be explained. This comment isn't that.
- Sure, but if you are sharing with a general audience (where people aren't necessarily coming from your own domain) it's a good idea to make the writing accessible.
My first thought would have been Band-Pass Filter, which is also a filter potentially related to computer systems.
I work in an industry with a lot of Three-Letter Acronyms (TLAs) and eXtended Three-Letter Acronyms (XTLAs) (sometimes known as Four-Letter Acronyms (FLAs)), and there they are often overloaded in their meanings. So in my experience, being clear about the definition is helpful to readers so they can immediately understand the document without having to triangulate meanings from the rest of the document.
- But again, this isn't intended for a general audience.
Anyone who might need this would already know what BPF is. And anyone who isn't familiar with the term BPF in this context wouldn't be the target audience for this.
It's also worth noting that BPF isn't ever referred to in it's non-acronym form. Literally no-one in the field calls this "Berkeley Packet Filter". Just like nobody calls PHP "PHP: Hypertext Processor" (or whatever backronym they've decided on this week), nor SQL as Structured Query Language. The name for this technically literally is just referred to as "BPF".
So while I agree with your point in general -- it's not really a fair complaint in this specific occurrence.
- Well, it's posted on Hacker News which is a general audience including people working on electrical/signal/audio/radio-frequency systems among other things (not just low-level network code) where BPF has a different meaning, so I think disambiguation is appropriate.
- > Well, it's posted on Hacker News which is a general audience including people working on electrical/signal/audio/RF systems among other things (not just low-level network code) where BPF has a different meaning, so I think disambiguation is appropriate.
And it's been explained on HN exactly what it is. So problem solved.
By the way, I noticed you didn't follow your own recommendation for the "RF" acronym in your comment. Nor "NASA" in your first reply. Perhaps you should check your own comments before you criticize others for doing the same.
Maybe you should also leave a comment in the Mullvad story (currently #1 on HN) that nobody has explained the VPN acronym there. Likewise for the threads where people reminisce about BASIC, of which there have been many lately. They're also only obvious if you already know the subject matter.
- Thanks for the catch! Fixed :-)
- Nitpick: eBPFs aren’t just for low level networking manipulations but also lots of other parts of the kernel. In fact the expanded version of the acronym is slightly confusing as it sort of implies being entirely to do with networking, which it isn’t.
- To be honest BPF is one of those acronyms that I think might be more recognizable as the acronym than what it actually stands for.
Not quite as much as Radar or Laser, but halfway there.
- And today I learned this has a name, anacronym! An acronym that has become so common, it's treated as a word instead of an acronym.
- Spelling out Berkeley Packet Filter doesn't really tell you anything about what eBPF actually is.
- What's NASA :)
- ~Well, if you asked a randomly-chosen person (technical or non-technical) they would probably say NASA is "the organization that does the space things" — it's pretty well-known~
~On the other hand, BPF means different things in different domains, and isn't ubiquitous in the same way~
Edit: I should have written it out, that's on me :-)
- 'berkeley packet filters', these days 'extended berkeley packet filters', are little program snippets you can inject into the linux kernel from userland programs to run logic on hooks on various events without the need to switch back to userspace.
- It's a low level networking interface https://en.wikipedia.org/wiki/Berkeley_Packet_Filter
- BPF == 'Berkeley Packet Filter'
Here you go: https://github.com/pratyushanand/learn-bpf
- The readme is an immediate giveaway of sloppiness
- try adding bindings to it from javascript and using it to render jsx in the terminal
- after going through the docs this looks quite useful, but I'd prefer if the AI features were optional.
assuming it's your project here is some unsolicited feedback:
(1) imprint missing, no idea where the company is operating based on name or tld, cannot rule out it is in adversary country
(2) not a fan of curl | sh, looks way more professional if some prebuilt packages for common distros are also offered. maybe remove the yellow box and add some distro logos "available on your favorite distro"
(3) On landing page I think the last section with the cost comparison should actually be at the very top. No sysadmin wants to have AI chat on their machines. The cost comparison chart shows well-known tools that every sysadmin knows (splunk etc), and directly relates yeet to it - this is very good.
(4) the main landing page hero text is not really explanatory - linux ops is a big term, and there was not a lot of info I got out of it. Further down there is "yeet gives you kernel level visibility with featherweight overhead. Nothing gets dropped.", which I'd personally prefer. Maybe instead of "yeet is a JavaScript runtime for Linux Ops." use something like "yeet is a Javascript runtime for your linux kernel".
Generally the sysadmins I know are not looking for AI chats or agent toolkits, and right now these are "features" that might make people close the tab. But sysadmins want to easily get custom analytics and reduce SaaS costs, these features are looked for.
Maybe it makes sense to more clearly split up the "specialized Javascript for linux Kernel" thing from the AI features. No manager bats an eye if I install a new Javascript runtime that allows better LOCAL-FIRST (!) linux kernel analytics, but a lot of explanation needs to be done if there are "agents" or "AI chats" which can potentially exfiltrate data.
- [dead]
- [dead]
- [flagged]