LLM image processing doesn’t work the same way reverse image lookup does.

Tldr explanation: Multimodal LLMs turn pictures into a thousand 200-500 or so words tokens, but reverse image lookups create perceptual hashes of images and look the hash of your uploaded image up in a database.

Much longer explanation:

Multimodal LLMs (technically, LMMs - large multimodal models) use vision transformers to turn images into tokens. They use tokens for words, too, but these tokens don’t also correspond to words. There are multiple ways this could be implemented, but a common approach is to break the image down into a grid, then transform each “patch” of a specific size, e.g., 16x16, into a single token. The patches aren’t transformed individually - the whole image is processed together, in context - but it still comes out of it with basically 200 or so tokens that allow it to respond to the image, the same way it would respond to text.

Current vision transformers also struggle with spatial awareness. They embed basic positional data into the tokens but it’s fragile and unsophisticated when it comes to spatial awareness. Fortunately there’s a lot to explore in that area so I’m sure there will continue to be improvements.

One example improvement, beyond improved spatial embeddings, would be to use a dynamic vision transformers that’s dependent on the context, or that can re-evaluate an image based off new information. Outside the use of vision transformers, simply training LMMs to use other tools on images when appropriate can potentially help with many of LMM image processing’s current shortcomings.

Given all that, asking an LLM to find the album for you is like - assuming you’ve given it the ability and permission to search the web - like showing the image to someone with no context, then them to help you find what music video - that they’ve never seen, by an artist whose appearance they describe with 10-20 generic words, none of which are their name - it’s in, and to hope there were, and that they remembered, the specific details that would make it would come up in the top ten results if searched for on Google. That’s a convoluted way to say that it’s a hard task.

By contrast, reverse image lookup basically uses a perceptual hash generated for each image. It’s the tool that should be used for your particular problem, because it’s well suited for it. LLMs were the hammer and this problem was a torx screw.

Suggesting you use - or better, using a reverse image lookup tool itself - is what the LLM should do in this instance. But it would need to have been trained to think to suggest this, capable of using a tool that could do the lookup, and have both access and permission to do the lookup.

Here’s a paper that might help understand the gaps between LMMs and tasks built for that specific purpose: https://arxiv.org/html/2305.07895v7

Thank you! That gives me a starting point that should be easy to look up!

From the blog post referenced:

We do not provide evidence that:

AI systems do not currently speed up many or most software developers

Seems the article should be titled “16 AI coders think they’re 20% faster — but they’re actually 19% slower” - though I guess making us think it was intended to be a statistically relevant finding was the point.

That all said, this was genuinely interesting and is in-line with my understanding of the human psychology that’s at play. It would be nice to see this at a wider scale, broken down across different methodologies / toolsets and models.

Why is 255 off limits? What is 127.0.0.0 used for?

To clarify, I meant that specific address - if the range starts at 127.0.0.1 for local, then surely 127.0.0.0 does something (or is reserved to sometimes do something, even if it never actually does in practice), too.

Advanced setup would include a reverse proxy to forward the requests from the applications port to the internet

I use Traefik as my reverse proxy, but I have everything on subdomains for simplicity’s sake (no path mapping except when necessary, which it generally isn’t). I know 127.0.0.53 has special meaning when it comes to how the machine directs particular requests, but I never thought to look into whether Traefik or any other reverse proxy supported routing rules based on the IP address. But unless there’s some way to specify that IP and the IP of the machine, it would be limited to same device communications. Makes me wonder if that’s used for any container system (vs the use of the 10, 172.16-31, and 192.168 blocks that I’ve seen used by Docker).

Well this is another advanced setup but if you wanted to segregate two application on different subnets you can. I’m not sure if there is a security benefit by adding the extra hop

Is there an extra hop when you’re still on the same machine? Like an extra resolution step?

I still don’t understand why .255 specifically is prohibited. 8 bits can go up to 255, so it seems weird to prohibit one specific value. I’ve seen router subnet configurations that explicitly cap the top of the range at .254, though - I feel like I’ve also seen some that capped at .255 but I don’t have that hardware available to check. So my assumption is that it’s implementation specific, but I can’t think of an implementation that would need to reserve all the .255 values. If it was just the last one, that would make sense - e.g., as a convention for where the DHCP server lives on each network.

Why is 255 off limits? What is 127.0.0.0 used for?

PSTN is wiretapped.

It’s a good thing that the website itself supports sending and receiving alerts, then.

Current generation iPad Pros and Airs have the same processing power as Apple Silicon Macs. That’s more than enough for Blender. Even the base iPad and the iPad Mini likely have enough processing power - though I don’t think the base iPad has enough RAM.

Does mirroring a screen (or adding a screen) from a computer or connecting to a computer via remote desktop count?

if everyone thought like you no one would create digital media

This is obviously incorrect.

When did Democrats have a supermajority in both the House and the Senate?

I thought Hue bulbs used Zigbee?

The up arrow moves through the letters, e.g., A->B->C. The down arrow moves to the next character in the sequence, e.g., C->CA->CAA. If you click past the correct letter, you’ll have to click all the way through again. And if you submit the wrong letter, you have to start all over (after it takes twenty seconds attempting to connect with the wrong password and then alerts you that it didn’t work, of course).

Fair point, I should have asked about commercial games in general

That said I didn’t mean that the game studio itself would do the AI training and own their models in-house; if they did, I’d expect it to go just as poorly as you would. Rather, I’d expect the model to be created by an organization specialized in that sort of thing.

For example, “Marey” is one example I found of a GenAI model that its creators are saying was trained ethically.

Another is Adobe Firefly, where Adobe says they trained only on licensed and public domain content. It also sounds like Adobe is paying the artists whose content was used for AI training. I believe that Canva is doing something similar.

StabilityAI is also doing something similar with Stable Audio 2.0, where they partnered with a music licensing company, AudioSparx, to ensure that artists are compensated, AI opt outs are respected, etc…

I haven’t dug into any of those too deep, but they seem to be heading in the right direction at the surface level, at least.

One of the GenAI scenarios that’s the most terrifying to me is the idea of a company like Disney using all the material they have copyright for to train their own, proprietary GenAI image, audio, and video tools… not because I think the outputs would be bad, but because of the impact that would have on creators in that industry.

Fortunately, as long as copyright doesn’t apply to purely AI generated outputs, even if trained entirely on your own content, then I don’t think Disney specifically will do this.

I mention that as an example because that usage of AI, regardless of how ethically the model was trained, would still be unethical, in my opinion. Likewise in game creation, an ethically trained and operated model could still be used unethically to eliminate many people’s jobs in the interest solely of better profits.

I’d be on board with AI use (in game creation or otherwise) if a company were to say, “We’re not changing the budget we have for our human workforce, including for contractors, licensed art, and so on, other than increasing it as inflation and wages increase. We will be using ethical AI models to create more content than we otherwise would have been able to.” But I feel like in a corporate setting, its use is almost always going to result in them cutting jobs.

Are you okay with AAA studios using GenAI that was trained only on licensed works?

Depends on your e-reader! If you have a Kindle, Kobo, or Nook, yes, that’s true. However:

Boox has e-readers that run Android and you can install Hoopla. The Palma 2 is phone sized which is great. The Page, Leaf2, and Go 7 are all in the 7” form factor, plus they have 6” versions. And they have tablet sizes, too. They have both traditional black&white and color e-ink displays.

I have the Boox Air 3C and the original Palma and both are great. I’ll likely get a Boox as my next standard sized e-reader, too (whenever I replace my Kindle Oasis). Though unless the technology drastically improves before then, it’ll be one with a black and white screen. (The color is nice in the tablet sizes, though, especially for comics from Hoopla.)

Some other options that I’m less familiar with include:

• Bigme has Android 7” color e-readers, as well as tablets and e-ink smartphones.
• Meebook has e-readers that run Android (and Android e-ink tablets)
• The MuSnap Aura C is a 10” Android e-ink tablet
• XPPen has an 11” Android e-ink tablet

Okay, and? What nontechnical user cares enough to use it specifically when they could use Microsoft Office, Google Docs, Polaris Office, MobiOffice, WPS Office, Collabora, etc., instead?

But do nontechnical users care about the “missing” features? A lot of nontechnical users prefer simpler apps.

There is a version of Blender that was made for Android. It’s quite old, though. But if you’re competent enough with Blender that you’ve memorized all its keyboard shortcuts and workflows, you’re likely technical enough to get it working via Termux. But if not, Nomad Sculpt (on both iOS and Android), SpaceDraw (Android only), and several other apps can serve the same purposes.

Not sure why you listed video editing software and two different specific video editors, but Android and iOS both have Lumafusion. I’m sure there are other decent editors but I haven’t used them because Lumafusion is great. iPads do have DaVinci Resolve, though, for what that’s worth. If you care about using a FOSS video editor then you should care enough to install it via Termux. But let’s be real, most nontechnical users are probably happy using CapCut.

DJ software - Cross DJ is free. There are other alternatives. And there are web based DJ software apps like YouDJ.

It’s incredibly compatible. Capitalists want laborers to work hard. It encourages laborers to work hard so they can one day be capitalists themselves.

It also encourages them to vote for politicians who don’t serve them, but politicians, because someday they’ll benefit from their pro-business policies.

The American Dream is capitalist propaganda, not anticapitalist.

The products currently on the marketplace have architectures that are far more sophisticated than just an LLM. Even something as simple as “Deep Research,” which both Anthropic and Claude have available, is using multiple interconnected systems to provide a single response.

Consider Agentic AI, like Claude Code, where they’re using tools, analyzing the results of those tools, iterating, possibly calling out to MCP servers to do other things, etc… The tools allow them to do things like read or modify files in the working directory, execute programs (i.e., your linter, installing dependencies, running your app), querying against your app itself, and so on.

And of course note that the single “Claude” box in that diagram has an architecture that’s more sophisticated than just being an LLM. At minimum, consumer facing LLMs generally have a supervisor that censors problematic inputs and outputs; this doesn’t make the system more competent but the same concept can be applied to any other sort of transparent wrapper.

It seems to me that we already have consumer systems that are doing what you described, and we’re already working on enhancing their architectures further.