Extending ActivityPub: A Recipe

Occasionally there are online discussions about using schema.org vocabularies with ActivityPub (AP) and Activity Streams 2.0 (AS2). There are many ways to do this, but this article will explore one approach using a schema.org Recipe type. The information is focused on AP/AS2 messaging rather than user interface considerations related to displaying the recipe. Although the discussion touches on some technical topics related to JSON-LD, the result is valid for software doing either plain JSON or JSON-LD processing.

NOTE: This article is for experienced ActivityPub developers. It assumes you are familiar with the AP/AS2 specifications and have some working knowledge of JSON-LD. Given the variability of ActivityPub implementations, these techniques are not guaranteed to work with all ActivityPub implementations. Interoperability challenges are discussed at the end of the article.

The Recipe

We'll start with an example recipe (from the schema.org site) as shown below:

{
    "@context": "https://schema.org",
    "@type": "Recipe",
    "author": "John Smith",
    "cookTime": "PT1H",
    "datePublished": "2009-05-08",
    "description": "This classic banana bread recipe comes from my mom -- the walnuts add a nice texture and flavor to the banana bread.",
    "image": "bananabread.jpg",
    "recipeIngredient": [
        "3 or 4 ripe bananas, smashed",
        "1 egg",
        "3/4 cup of sugar"
    ],
    "interactionStatistic": {
        "@type": "InteractionCounter",
        "interactionType": "https://schema.org/Comment",
        "userInteractionCount": "140"
    },
  "schema:name": "Mom's World Famous Banana Bread",
    "nutrition": {
        "@type": "NutritionInformation",
        "calories": "240 calories",
        "fatContent": "9 grams fat"
    },
    "prepTime": "PT15M",
    "recipeInstructions": "Preheat the oven to 350 degrees. Mix in the ingredients in a bowl. Add the flour last. Pour the mixture into a loaf pan and bake for one hour.",
    "recipeYield": "1 loaf",
    "suitableForDiet": "https://schema.org/LowFatDiet"
}

AS2 is very flexible so this document is almost conformant already. A few differences with a typical AS2 document are:

• There is no https://www.w3.org/ns/activitystreams in the @context.
• The @type value is not a known AS2 type. (Note that @type is aliased to type in ActivityPub)

The following sections will discuss these issues and how to address them.

Adding an Activity Streams 2.0 Context

It may be surprising, but AS2 doesn't require the normative JSON-LD context URL to be included in the document given certain preconditions are satisfied.

When a JSON-LD enabled Activity Streams 2.0 implementation encounters a JSON document identified using the " application/activity+json" MIME media type, and that document does not contain a @context property whose value includes a reference to the normative Activity Streams 2.0 JSON-LD @context definition, the implementation must assume that the normative @context definition still applies. -- [Section 2.1, Activity Streams 2.0 Core](Activity Streams 2.0 (w3.org))

Unfortunately, it's unclear what "assume the context still applies" means. One interpretation is that a JSON-LD consumer (versus a plain JSON consumer) should add the AS2 context before doing JSON-LD expansion. (The JSON-LD expansion algorithm expands the property names to their fully qualified names and associates other metadata with the property, like whether it's a URI (identifier) or a value with an optionally specified type. For example, name becomes https://www.w3.org/ns/activitystreams#name.)

Even with the assumption that the AS2 context applies, it's still ambiguous where the normative context should be included relative to other existing context information. JSON-LD has a "last definition wins" policy so the ordering matters.

For this example, we could include the AS2 context either before or after the schema.org context before JSON-LD expansion. Placing it before the schema.org context will be a problem though. The AS2 specification states that:

Implementations may augment the provided @context with additional @context definitions but must not override or change the normative context. [Section 2.1, Activity Streams 2.0 Core](Activity Streams 2.0 (w3.org))

The schema.org JSON-LD context defines terms like image and name that would override the AS2 terms. Since this isn't allowed, we'll include the AS2 context after the schema.org context. The context now looks like this:

{
    "@context": [
      "https://schema.org", 
      "https://www.w3.org/ns/activitystreams"
    ],
    . . .

Adding an Activity Streams 2.0 Type

The @type property (AS2 aliases it to type) is an issue. AS2 doesn't actually require the property (no properties are required), but if it's there, AS2 requires it to refer to an Object or a Link. A JSON-LD document can specify multiple types so, we can add Object to the list of types. However, a more specific subclass of Object, like Document or Note may be better.

Now, with the added type, the document looks like:

{
    "@context": [
      "https://schema.org", 
      "https://www.w3.org/ns/activitystreams"
    ],
    "@type": ["Recipe", "Document"],
    . . .

When expanded, the type values become:

Compact Type	Expanded Type
Recipe	http://schema.org/Recipe
Document	https://www.w3.org/ns/activitystreams#Document

No now, theoretically, and schema.org consumer will recognize the Recipe type and any AP/AS2 consumer will recognize the Document type. Unfortunately, it's not as simple as that since ActivityPub servers typically only support a small subset of the AS2 types. Interoperability issues will be discussed in more detail later in this article.

At this point, we'll change the @type property name to the AS2 alias, type, to improve interoperability with servers looking for the string "type".

Property Name Conflicts

We now have a conformant AS2 document. However, there is still an issue. We have changed the expanded keys of name and image to use the AS2 prefix since AS2 overrides the schema.org terms. They previously used the schema.org prefix. Depending on the application, this might be a problem.

One way to fix the issue, from a JSON-LD processing perspective, is to use a prefix for the schema.org properties that have been overridden by AS2.

For example we could change…

    "image": "bananabread.jpg",

to

    "http://schema.org/image":  "bananabread.jpg",

or, since a "schema" prefix alias is defined, it could be

    "schema:image":  "bananabread.jpg",

which would expand the longer prefix shown earlier.

This is fine for software doing JSON-LD processing, but it may cause issues for plain JSON processing since that code would typically be looking for a key name image rather than schema:image. However, if a recipe sharing interoperability profile existed, it may specify that name and image will be schema:name and schema:image. If so, it's not a problem. Any plain JSON processing code can use that information to look for those key names. The other option is to let them expand to AS2 terms and not change the property names. An interoperability profile should specify these details.

Using the AS2 names, the final document looks like:

{
    "@context": [
      "https://schema.org", 
      "https://www.w3.org/ns/activitystreams"
    ],
    "type": ["Recipe", "Document"],
    "author": "John Smith",
    "cookTime": "PT1H",
    "datePublished": "2009-05-08",
    "description": "This classic banana bread recipe comes from my mom -- the walnuts add a nice texture and flavor to the banana bread.",
    "image": "bananabread.jpg",
    "recipeIngredient": [
        "3 or 4 ripe bananas, smashed",
        "1 egg",
        "3/4 cup of sugar"
    ],
    "interactionStatistic": {
        "@type": "InteractionCounter",
        "interactionType": "https://schema.org/Comment",
        "userInteractionCount": "140"
    },
    "name": "Mom's World Famous Banana Bread",
    "nutrition": {
        "@type": "NutritionInformation",
        "calories": "240 calories",
        "fatContent": "9 grams fat"
    },
    "prepTime": "PT15M",
    "recipeInstructions": "Preheat the oven to 350 degrees. Mix in the ingredients in a bowl. Add the flour last. Pour the mixture into a loaf pan and bake for one hour.",
    "recipeYield": "1 loaf",
    "suitableForDiet": "https://schema.org/LowFatDiet"
}

This document could be used with existing AS2 Activity types like Create, Announce and so on.

Microblogging Interoperability

The document I've presented could be used for AP interoperability between recipe-sharing applications. However, it's not going to work with microblogging platforms like Mastodon, which expects objects to be some variant of an AS2 Note with specific properties.

We could use Note instead of Document (or in addition to it) in the types and Mastodon would detect that type. I haven't tried it yet, but that's probably not enough. However, the Mastodon document does say it will use the name property to generate status content.

The transformer uses content if available, or name if not, in order to generate status text. The url will be appended. The summary property will be used as the CW text. The icon will be used as a thumbnail. -- Mastodon ActivityPub Documentation

The icon usage sounds potentially interesting, but unfortunately it appears (from looking at the source code) that the icon is expected to be a "media attachment" rather than a top-level property in the AP Object. In any case, it's not looking for image (other than in actor profile documents).

Exploring Further

I recommend taking the initial and final versions of the Recipe document and exploring it using the JSON-LD Playground. Try pasting in the JSON and look at the "Expanded" tab. This will show you how the property names have been expanded.

If you see a name starting with _:, it means it is a "blank node" (an RDF technical term). It means the object has no defined identifier. This might be fine or it might be an indication of an error in the context definition.

Although an application doing plain JSON processing will not be doing expansion, it's good to see what happens, in that case, to ensure the context is correct for applications that do JSON-LD processing.