mod_http_file_share XEP-0363: HTTP File Upload

Allow users to share files, such as pictures, videos and audio clips, by uploading them to the server, giving them an URL to pass on to one or more recipients.

This module is not yet included with Prosody, it’s currently only available in trunk builds and will be in the next major release.

Details

mod_http_file_share implements XEP-0363: HTTP File Upload and supports using either Prosody’s built-in HTTP server or an external upload service.

Getting started

Simply add the following at the end of your config:

Component "upload.example.org" "http_file_share"

That’s it! Restart Prosody and try to share something!

HTTP configuration

The HTTP aspects of this module (HTTP host, path, etc.) are fully configurable using Prosody’s standard HTTP configuration. By default the module will use the path /file_share. To find the URL that the module is currently using, you can run (assuming you have mod_admin_shell enabled):

prosodyctl shell module info http_file_share

Alternatively, check your log file for the informational messages logged by mod_http_file_share when it is first loaded.

Component and HTTP domains

The domain specified in your Component line should ideally be a direct subdomain of your VirtualHost. Sometimes this isn’t possible (e.g. you may have more than one VirtualHost sharing the same component) - in that case see the ‘Discovery’ section below for the necessary configuration.

You generally do not need a DNS record or certificate for the component domain, but you do need one for the HTTP domain. By default the component and HTTP domains are the same (e.g. upload.example.org in the example above). But you can control the HTTP domain (the one that needs DNS and certificate) either by setting http_host or using a reverse proxy and configured http_external_url.

For example, to avoid an additional DNS record and certificate, you may choose to set http_host to your primary domain (e.g. http_host = "example.com") or use a reverse proxy to handle the HTTP for that domain.

Discovery

If your component domain (e.g. upload.example.org) is a direct subdomain of your VirtualHost that your users accounts are on (e.g. example.org), your users’ clients will automatically discover and use your upload service.

If, however, your component domain is not a direct subdomain (e.g. upload.example.net or upload.foo.example.org) then you will need to specificaly add that the component to your VirtualHost’s disco_items so that clients can find it.

Manual configuration to allow clients on a VirtualHost to discover an upload component that is not a direct subdomain:

VirtualHost "chat.domain.example"
disco_items = {
    { "upload.example.org", "file sharing service (this name is optional)" },
}

Larger files

Users wanting to share even larger files? The size limit can be increased with the http_file_share_size_limit setting.

Component "upload.example.org" "http_file_share"
http_file_share_size_limit = 16*1024*1024 -- 16 MiB

Daily quotas

The daily quota is a limit that prevents users from uploading more files if more than a certain amount of bytes have been uploaded during the last 24 hours.

Component "upload.example.org" "http_file_share"
http_file_share_daily_quota = 100*1024*1024 -- 100 MiB per day per user

Thus with the above configuration and one week retention, each user could store up to 700 MiB each.

Once this limit has been reached they must wait until at least one upload to become more than 24 hours old.

Global quota

Since it’s not always easy to estimate how much storage space will be needed, i.e. because the number of users may change which would increase the total theoretical usage by another 700 MiB (in the example from the previous section).

Component "upload.example.org" "http_file_share"
http_file_share_global_quota = 1024*1024*1024 -- 1 GiB total

With such configuration, the total size of all uploaded files will not be allowed to exceed 1 GiB.

Access

Any local user on the same Prosody instance is allowed to use the upload service by default. The http_file_share_access setting can be specified to limit access to certain users or hosts, including hosts on other servers. Be careful!

Component "upload.example.org" "http_file_share"
http_file_share_access = {
    "filesharingenthusiast@example.net", -- this specifc user
    "example.org", -- anyone with a @example.org address
}

Restricting file types

Only want to serve images? Say so with the http_file_share_allowed_file_types setting.

Component "images.example.org" "http_file_share"
http_file_share_allowed_file_types = { "image/*" }

Using a separate HTTP upload service

To use an external upload service, specify the base URL of via http_file_share_base_url and configure a strong shared secret via http_file_share_secret, which must also be specified in the configuration of the external upload service.

Component "upload.example.org" "http_file_share"
http_file_share_secret = "U2VjcmV0VG9rZW4wMzYz"
http_file_share_base_url = "https://share.example.com/upload.php/"

Configuration

All settings

Setting type default note
http_file_share_size_limit number 10*1024*1024 10 MiB
http_file_share_daily_quota number 10x the size limit 100 MiB
http_file_share_allowed_file_types set {} Access control
http_file_share_safe_file_types set {"image/*","video/*","audio/*","text/plain"} Safe to show in-line in e.g. browsers
http_file_share_expires_after number 7 * 86400 One week
http_file_share_access set {} Access control
http_file_share_base_url string unset Base URL of external upload service
http_file_share_secret string random External upload service

External upload protocol details

The external protocol works by passing various details in a JSON Web Token in an Authorization header.

Example payload:

{
   "exp" : 1612113460,
   "expires" : 1612717960,
   "filename" : "picture.png",
   "filesize" : 7168,
   "filetype" : "image/png",
   "iat" : 1612113160,
   "slot" : "RVbKAgWPcrUYAodj",
   "sub" : "user@example.com"
}

This is normally signed with the HS256 algorithm (HMAC-SHA-256). The fields are as follows:

slot
Unique identifier for this upload.
iat
UNIX timestamp of when the token was issued.
exp
UNIX timestamp after which the token should be considered expired.
sub
Identity of the uploader, for per-user quotas.
filename
Name of the file being upload.
filesize
Size of the file, in bytes. Ensure this matches the size of the upload.
filetype
MIME type of the file. Ensure this is the content type the file is served with.
expires
UNIX timestamp after which the file should be considered expired.

Example request (line breaks for readability):

PUT /base_path/RVbKAgWPcrUYAodj/picture.png
Content-Type: image/png
Content-Length: 7168
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
    eyJmaWxlbmFtZSI6InBpY3R1cmUucG5nIiwic2xvdCI6IlJWYktBZ1dQ
    Y3JVWUFvZGoiLCJmaWxlc2l6ZSI6NzE2OCwiZXhwIjoxNjEyMTEzNDYw
    LCJzdWIiOiJ1c2VyQGV4YW1wbGUuY29tIiwiZmlsZXR5cGUiOiJpbWFn
    ZS9wbmcifQ.7iiwBKcxCSx-UJQaVmj3CpYvIAve4SCmGJwiw4Fmjgc

< FILE CONTENT FOLLOWS >

What data is stored?

For each file uploaded, the following information is stored:

  • File metadata; name, size and file type provided by the client.
  • File data itself.
  • Time when the upload slot was created.
  • Address of the user who requested the upload slot.

File data and metadata is obviously required to download the file again.

User address is needed to apply upload quotas.