IPFS uploads

submitted by Gangster from self.IdeasForSaidIt

Hi,

Not a programmer, but a tech-related professional. I think this should be possible, and I might be willing to fund a bounty for it.

To submit a picture/video/other, a Saidit user currently has to upload it on an offsite host, then submit the link to Saidit. These links can go down or be censored, and it's an extra step for the user.

Reddit implemented their own hosting (i.redd.it) to reduce reliance on third-party hosts, and save the user that one step. However, the problem of censorship is still there.

I would be interested in a third way: when a user clicks 'submit link', they can upload an image or video to IPFS. This would ensure permanence and censorship-resistance, and reduce reliance on hosting sites without increasing hosting costs for Saidit.

This is possible - https://ipfs.pics/ currently allows drag-and-drop uploads in the browser. Code here: https://github.com/ipfspics/ipfspics-server

Feedback is welcome. How long would it take developers to implement this?

all 24 comments
sorted by:
top
newinsightfulfun

[–]friend 5 insightful - 2 fun5 insightful - 1 fun6 insightful - 2 fun -  (1 child)

If you use spee.ch as your image host your content will be served over IPFS, and the metadata will be hosted on the LBRY blockchain.

[–]Gangster[S] 1 insightful - 2 fun1 insightful - 1 fun2 insightful - 2 fun -  (0 children)

I did not know this! Thanks!

[–]magnora7 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (7 children)

That's a very interesting idea! I like it.

From a programming perspective, it would probably take between 50 to 500 hours to accomplish. However just getting it working for jpg images only might be a good first step, and that might be possible in 10 to 100 hours of programming time, I'd estimate.

I'll think about this idea some more, I do like it. We've previously had the idea of hosting the whole site on IPFS, but just hosting the media content would be an interesting and useful intermediate step as well. And hosting jpgs that are link posts would be a good very first step, sort of a proof-of-concept.

We've never paid any programmers before, so I'm not sure how the bounty would work (or if it's even necessary tbh.) I'll think about this some more. Good suggestions!

[–][deleted] 3 insightful - 3 fun3 insightful - 2 fun4 insightful - 3 fun -  (1 child)

Hahahaha this sounds like a me estimate, 50 to 500 hours.

[–]magnora7 2 insightful - 2 fun2 insightful - 1 fun3 insightful - 2 fun -  (0 children)

Lol it's just the truth, I simply cannot peg it down any more than an order of magnitude, and hardly even that. You and I both know it well from experience haha

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

Great!

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

Would this help?

https://github.com/ipfspics/ipfspics-server#can-i-install-ipfspics-on-my-own-server

[–]magnora7 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

Yes that's very helpful, thank you. This particular one only does pics though, so it'd be great for jpgs but I'm not sure we can expand it to videos or other stuff. But we may be able to.

The hackernews thread linked to in the readme is pretty good on discussing the limitations of this technology: https://news.ycombinator.com/item?id=10436792

The only way a hash becomes distributed on the network is for another party to request that hash. Even then, I believe that the data only exists on that party's system. That is, if both you and the requester leave the network, the data is gone.

I think on saidit it should work like things are fetched from the normal server, and then they simultaneously have the opportunity to be fetched from the users helping serve the IPFS system. So it gives redundancy without any loss of speed (and maybe with the potential to increase speed, even).

I guess also our thumbnails are stored on-server. So there are downsized copies of the images still stored for the thumbnails next to articles and links to images. This is similar to i.reddit. We've had a time where the external server went down and we had an internally-hosted link where it was still visible, but it was downsized.

Anyway, I'm going to look in to this more, thanks for the ideas. I'm not quite sure how the part about users hosting would work. Does hosting require external software in IPFS, or is it easily doable through the web browser just by visiting a link? I assume it's the latter. But then how long do they continue to host? Just until they leave the page with the picture on it? These are the little details that need figuring out based on how it's currently designed...

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

Here's one for videos: https://github.com/download13/ipfstube

[–]magnora7 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Nice, that's exciting.

[–][deleted] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (9 children)

Let's say SaidIt ran a copy of ipfspics and we offered it here for image uploads. How is that any more decentralized than SaidIt just offering image uploads?

The catch seems to be that ipfspics is maintaining a list of ipfs hosts which they will mirror the uploads to. So unless there's a third party willing to mirror all SaidIt uploaded images, then the whole thing still seems centralized to me. Maybe I'm missing something.

We make sure that everything you upload to the official ipfs.pics instance is distributed on multiple computers.

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (4 children)

I upload things to IPFS frequently, and often come back weeks later to find they're still up, even when my computer is offline and I haven't shared them widely.

Who's seeding them in these cases? I have no idea. Might be some big tech company that scours IPFS and autoseeds everything.

[–][deleted] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

That's what I'm getting at, who is making it happen at the end of the day? I don't know a lot about IPFS, but it seems that the ipfs.pics project is footing the bill and ensuring that all of their images stay up (centralization).

But even if IPFS is not a silver bullet for this SaidIt use case, other advantages may make it worth it and mean that we should use it as our approach. Hopefully an expert will pop up in here and set us straight.

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (2 children)

I have two basic motivations for the proposal:

  1. Low-cost, decentralised hosting. As you say, this raises some questions, and someone needs to be there to hold the baby.

  2. Censorship-resistance, which is the bigger motivation

[–][deleted] 3 insightful - 2 fun3 insightful - 1 fun4 insightful - 2 fun -  (1 child)

Thank you for the suggestion! I don't mean to doubt your motivations. Just trying to figure out all of this new tech. And the reasons you listed are great ones and valuable to SaidIt.

[–]Gangster[S] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (0 children)

Right. Part of the ethos here is censorship-resistance, I believe.

[–]magnora7 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

How is that any more decentralized than SaidIt just offering image uploads?

I am trying to wrap my head around this too. I think it would give us the option of decentralization, so if all of saidit and all our backups somehow got destroyed, then the IPFS-hosted data would have a chance to survive.

But at the end of the day I don't know how a user helps host our data. Do they sit on a certain page? Or can they browse the site while sharing? Does it require a browser plugin or download to help peer-share? These are things I still don't understand about it.

[–][deleted] 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (2 children)

That's it, that's where I got stuck too. I did a little more reading, and I think that either we'd have to publish a list of every saidit related ipfs image in hopes that others would request/save/reseed them, or force share everything to another ipfs-pic instances (again requiring a third party that gives a shit).

[–]magnora7 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

I think the latter is what we'd want, or the list would somehow naturally accumulate with code from the site. But I still don't get what actions a user would have to take to actually help host the data, even if they wanted to.

[–][deleted] 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (0 children)

what actions a user would have to take to actually help host the data

If we only offered ipfs image links or a preference to use ipfs links instead, then people could run a local ipfs instance and this browser extension and then I think when they view the image they are downloading it to their local instance and hosting it for the next guy.

Aside from that, there's the full blown browser only ipfs implementation which can somehow get you swapping media with others directly (no extension, no server side local node, the browser is the node), like BitChute. However I see it's very alpha and they have browser crashing issues. That is some heavyweight js code to be running alongside a site like this too.

[–]fred_red_beans 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (3 children)

Anyone know how IPFS is going to deal with copyrighted material and the like?

[–]Gangster[S] 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

Same way TCP/IP does, or Bittorrent.

[–]fred_red_beans 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (1 child)

TCP/IP is a transmission protocol.

Bittorrent is also a communication protocol but does appear to have a Copyright Infringement Policy.

IPFS also has a way to Report Copyright Infringement on the IPFS Gateway Service.

But doesn't that go against the concept of the permanent interplanetary file system?

[–]Gangster[S] 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

That's what I'm getting at; IPFS is also a protocol.

The gateway isn't part of IPFS; it's just a convenient portal.