@volpeon *sigh*

People really love to hate any mention and any implementation of (Gen)AI without thinking about it for a few seconds.

Luckily, I switched to Vaultwarden a few months ago, so I don't have to deal with the "uhh but you're still using KeePassXC, you're pro ai!!!!!" shit. KeePassXC is great for what it is. I really liked using it. And I'll probably continue recommending it to people if it fits their needs.

@volpeon I think there is a difference, because code submitted by an outside contributor is generally understood to be potentially risky.
But being willing to use an AI coding tool *at all* implies a level of trust in the technology. And one of the big issue with this tech is a tendency to be "magical" in the mind of its advocates, which increases risk.
Whether this concern is valid or not is debatable, but these are definitely not the same situation

@charlotte @volpeon
I mean it's pretty close to the second one tbh. Like it doesn't have malice cause it's just an approximation of a high-context markov chain (no feelings), but like; by design it creates code that is very good looking, but also kinda bad, and completely lacks thought and understanding of the problems and edge cases that code solves.

@gimmechocolate @volpeon do we ban the use of copy paste due to the lack of awareness in that tool?

@SteffoSpieler @volpeon

I mean, there's the OG KeePass2, still maintained by one author who releases a zip of source code with every version instead of doing any modern github stuff.

To me, the overall UX ergonomics of KeePassXC and especially its information density per screen are lacking a lot in comparison.

@gimmechocolate @volpeon like ultimately it falls onto the developer to verify the work performed by the tools that they use, and to adjust the code accordingly

@errant From what I've seen from the developers, they're well aware of the risks and capabilities of AI. You aren't wrong that careless (non-)developers are too confident in AI, but this doesn't imply the inverse: that all users of AI are automatically careless. As long as they consistently demonstrate responsible use of it, I personally see no problem from a security standpoint.

@charlotte @volpeon
So like, first off, you're clearly responding to what you thought I'd say, and not what I actually said, because I didn't even mention lack of awareness.

Second, copy-paste is actually notoriously bad for creating errors, and it's typically considered good practice to use things like constants, macros, and functions instead. And that's again, ignoring the fact that copy-paste doesn't have any of the problems I mentioned in my post.

@Tvorsk @volpeon I liked the design of KPXC and I'm not a fan of the old look of KP2, but that's a thing of personal taste. If you like KP2, then that's absolutely awesome!

@SteffoSpieler @volpeon Yeah, kind of matter of taste for sure... just sayin' it's an option.
I believe it even runs under dotnet runtime on linux, although that'd be... probably not very convenient.

@charlotte @volpeon yeah the people on that github issue were bizarrely saying things like software is untestable, like you couldn't write a test to verify correctness because the nefarious ai can insert tricks that human cognition weaknesses cannot detect

@gimmechocolate @volpeon feelings imply awareness, no? but copy-paste doesn’t have feelings either.

and like yeah it does create issues. i am aware that it creates issues. that is specifically why i mention it. it is still an occasionally useful tool which can be leveraged by programmers. The person liable for (mis)use is the developer using the tool, and copy-paste errors are also something that may not be immediately obvious in a code review either

@volpeon@icy.wyvern.rip me trying to explain to people that just because it's open source doesn't mean it's automatically secure.

Just because you can read the source doesn't mean any single individual has complete understanding of the entire source code and even if that was possible, silence can be bought.

Open source fanatics are clueless sometimes.

@sun @charlotte @volpeon and the actual issue was some basic straightforward JSON import with tests

@volpeon yes, this.

@volpeon @errant AI is really easy to accidentally prompt in such a way that it goes off the rails, can make all sorts of mistakes and copyright issues (even subtle ones), and can do it at scale. Hypothetically it might be possible to use it in such a way to not trigger these issues, but surely all the checking and prerequisite expertise would nix most advantages of using it?

And it looks like for most AI PRs in KeepassXC, the person working with the AI ultimately approves the code (example: https://github.com/keepassxreboot/keepassxc/pull/12588)... hardly a rock solid review process. Usually, there's two humans in the loop.

@sitcom_nemesis @errant
> but surely all the checking and prerequisite expertise would nix most advantages of using it?

Sure, but why would this be a concern for anyone but the user themself? I'm sure I use things which other people may not like, such as VSCode or GNOME. Is it valid for them to tell me what to use and how?
> And it looks like for most AI PRs in KeepassXC, the person working with the AI ultimately approves the code... hardly a rock solid review process. Usually, there's two humans in the loop.

The way the AI is integrated in GitHub makes it a separate entity from the reviewer with an interaction workflow akin to iterating a PR with its author until it matches the project's standards. In both cases, the PR author — AI or human — is untrustworthy and the reviewer is trustworthy. There are also non-AI PRs where only one developer conducted the review, so there's no difference between AI and non-AI standards.

If this strikes you as flawed, then your concerns should lie with the review process itself.

@charlotte @volpeon
Yeah, but no feelings wasn't a criticism of LLMs, I was saying it does actually behave more or less like what you said but without the malice part. To be clear it's bad because it makes code that looks good, but isn't — it's just mashes up the semantic ideas from its training data without regard for how those different patterns will interact; the result will always *look* right, but won't necessarily be right, often for very subtle reasons, making it a poor tool that people simply shouldn't use. I wouldn't trust a project that put instructions on how to copy-paste from stackoverflow in their repo either.

As for the tool thing, like, I can see ways where a LLM could be a useful tool. For example, it's very annoying to me how the results from the LSP autocomplete are in some arbitrary order, and it could be nice to have an LLM step in and rank em!! It's a tool, but like, only as a part of a well-designed machine-learning pipeline, made by the people who understand the limits of the technology — not something used by just about everyone almost raw.

The whole thing is just annoying to me though — I've been interested in machine learning for over a decade now, and when I took my first course in it, I was warned about this exact scenario. There's a cycle of AI spring and winter, where the AI business leaders push the idea that these tools can do anything. People metaphorically saying "Uhm, when all you have is a hammer, actually you can solve all your problems by whacking everything with it.". Now I get to see what they were talking about firsthand and it's extremely frustrating to feel like I am shouting it from the rooftops and no one listens.

@sun @charlotte @volpeon i mean in my experience the pattern matching leading to outputting stuff that is very good at looking correct during ocular inspection but isn't correct is very real

@volpeon @errant

The way the AI is integrated in GitHub makes it a separate entity from the reviewer with an interaction workflow akin to iterating a PR with its author until it matches the project's standards. In both cases, the PR author — AI or human — is untrustworthy and the reviewer is trustworthy.

I'd argue that integrating AI into GitHub this way is part of the problem. It's not an agent - it's a word guessing machine with access to an API. It fundamentally doesn't think like a human, trustworthy or otherwise. We have methods of understanding context, intention and trustworthiness with other humans - AI strips that all away while still claiming to be analogous to a human. That's, in part, what makes it so risky.

It's one thing to allow AI code with the caveat that the human needs to take full responsibility (e.g. the Fedora guidelines), but that doesn't seem to be happening with KeepassXC. Hence the concern.