This looks good to me. For the data file mimetypes, would it make sense to distinguish between JSON and YAML (OPA supports both)?

There shouldn't be any issue using the OPA logo/icon in the registry. /cc @caniszczyk just to double check.

I think the convention here is use subtypes. SO you can say "this is data" and "this is formatted in YAML". eg.

application/vnd.cncf.openpolicyagent.data.v1+json
application/vnd.cncf.openpolicyagent.data.v1+yaml

Hey Gareth, et all. Great to see new scenarios.
I’m departing on a flight, so a quick response.
on the bundle, i’m assuming this is how you want registries to identify your artifact type using manifest.config.mediaType.
here’s what i’d suggest: application/vnd.cncf.openpolicyagent.config.v1+json
This declares the type, version, provides for a config and says the config is in json format. If you don’t need a config, you can send an empty json config file or should be able to specify dev/null, but i saw an error when using ORAS yesterday
for the rego and data files, are these layers
If so, i’d suggest application/vnd.cncf.openpolicyagent.data.layer.v1.json (or yaml)
the important part is to make it obvious it’s a layer type. If there are multiple files in the layer, using tar. If there are individual files, you can save on the compression and decompression and just reference txt, yaml, json, html, xml, whatEverNewFileFormat comes.
Here’s the bare bones we’re using for this week, prepping for rejekts and kubecon: https://github.com/Azure/acr/blob/master/docs/artifact-media-types.json
Are you planning to use OCI index to reference multiple artifacts, or just one OCI manifest?
I have more clarity on how we want to describe this in the distribution spec, so i’ll work on clearing up the inferences form all the discussion.
I’ll review more tonight as we’re taking off.

Thanks for the feedback Steve. Looks like we had the same thing for the top level config. Modified versions of the layer suggestions below:

This assumes a single layer per file, mainly to facilitate easy uploading with oras like so:

oras push localhost:5000/my-bundle-bundle:latest \
  .manifest:application/vnd.cncf.openpolicyagent.manifest.layer.v1.json \
  helpers.rego:application/vnd.cncf.openpolicyagent.rego.layer.v1.rego \
  deny.rego:application/vnd.cncf.openpolicyagent.policy.layer.v1.rego \
  samples.json:application/vnd.cncf.openpolicyagent.data.layer.v1.json \

I assume that later on also using application/vnd.cncf.openpolicyagent.policy.layer.v1.tar and application/vnd.cncf.openpolicyagent.data.layer.v1.tar for multiple data or policy files per layer would also make sense.

@jdolitsky I'd love your feedback on whether that makes sense?

I'd imagine writing a custom uploader with some domain knowledge using oras as a library, but looking to punt that to a little later.

Ahh, I see the confusion. You don't actually push a manifest by referencing a file. By pushing the --manifest-config with a mediaType, the OCI manifest is constructed for you. See the modified version below. Also, note the +rego vs. .rego

oras push localhost:5000/my-bundle-bundle:latest \
  --manifest-config ./config.json:application/vnd.cncf.openpolicyagent.config.v1+json \
  helpers.rego:application/vnd.cncf.openpolicyagent.rego.layer.v1+rego \
  deny.rego:application/vnd.cncf.openpolicyagent.policy.layer.v1+rego \
  samples.json:application/vnd.cncf.openpolicyagent.data.layer.v1+json 

If you don't need a config.json file, you should be able to pass in dev/null. I believe this is not currently supported in the ORAS cli, but we'll need to investigate and fix if not supported.

So, you don't actually need the

You don't actually push a manifest by referencing a file

There are two manifests :) One is the OCI manifest for the bundle as a whole, which I'd missed out from the oras command example. The other is the .manifest file (formatted in JSON) that is specific to OPA and defined the OPA bundle docs. The intention is to support storing that on it's own layer.

Also, note the +rego vs. .rego

Ah, the first draft of your blog post had eg. application/vnd.cncf.helm.values.layer.v3.yaml so I thought this had changed. But it's the right way round in the final published post. Swapping back.

Updates for both of those below:

And usage with Oras:

oras push localhost:5000/my-bundle-bundle:latest \
  --manifest-config ./config.json:application/vnd.cncf.openpolicyagent.config.v1+json \
  .manifest:application/vnd.cncf.openpolicyagent.manifest.layer.v1+json \
  helpers.rego:application/vnd.cncf.openpolicyagent.policy.layer.v1+rego \
  deny.rego:application/vnd.cncf.openpolicyagent.policy.layer.v1+rego \
  samples.json:application/vnd.cncf.openpolicyagent.data.layer.v1+json \

The world of manifests. This makes sense.
If you can do a PR to https://github.com/Azure/acr/blob/master/docs/artifact-media-types.json for the OCI manifest.config.mediaType and the short name, we can get this loaded up for next week.

@garethr your table of mediatypes + ORAS command in your last comment looks good to me, although seems the command has one extra media type than the table? It could maybe be the case that some bundle contain files/layers that others don't, and that's ok if the client knows how to handle it.

For some hints of using ORAS as a Go lib, you can take a look at the docs. We are still working on putting together a bunch of examples for copy-pasta goodness.

The following is how you might push an OPA bundle based on your spec:

package main

import (
	"context"
	"fmt"
	"io/ioutil"

	"github.com/containerd/containerd/remotes/docker"
	"github.com/deislabs/oras/pkg/content"
	"github.com/deislabs/oras/pkg/oras"
	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

const (
	// Remote reference
	// Command used to run a local test registry on port 5000:
	//   docker run --rm -it -p 5000:5000 registry
	RemoteRef = "localhost:5000/opa/mybundle:1.0.0"

	// OPA-specific file names
	// Command used to create dummy files created for test purposes:
	//   for f in ".manifest" "helpers.rego" "deny.rego" "samples.json"; do echo "hello world" > $f; done
	OpenPolicyAgentManifestFileName = ".manifest"
	OpenPolicyAgentRegoFileName     = "helpers.rego"
	OpenPolicyAgentPolicyFileName   = "deny.rego"
	OpenPolicyAgentDataFileName     = "samples.json"

	// OPA-specific media types
	OpenPolicyAgentConfigMediaType        = "application/vnd.cncf.openpolicyagent.config.v1+json"
	OpenPolicyAgentManifestLayerMediaType = "application/vnd.cncf.openpolicyagent.manifest.layer.v1+json"
	OpenPolicyAgentRegoLayerMediaType     = "application/vnd.cncf.openpolicyagent.rego.layer.v1+rego"
	OpenPolicyAgentPolicyLayerMediaType   = "application/vnd.cncf.openpolicyagent.policy.layer.v1+rego"
	OpenPolicyAgentDataLayerMediaType     = "application/vnd.cncf.openpolicyagent.data.layer.v1+json"
)

func check(e error) {
	if e != nil {
		panic(e)
	}
}

func getFileContents(fileName string) []byte {
	b, err := ioutil.ReadFile(fileName)
	check(err)
	return b
}

func main() {
	// Setup
	var contents []byte
	var layer ocispec.Descriptor
	var layers []ocispec.Descriptor
	resolver := docker.NewResolver(docker.ResolverOptions{})
	memoryStore := content.NewMemoryStore()

	// Add layers
	contents = getFileContents(OpenPolicyAgentManifestFileName)
	layer = memoryStore.Add(OpenPolicyAgentManifestFileName, OpenPolicyAgentManifestLayerMediaType, contents)
	layers = append(layers, layer)

	contents = getFileContents(OpenPolicyAgentRegoFileName)
	layer = memoryStore.Add(OpenPolicyAgentRegoFileName, OpenPolicyAgentRegoLayerMediaType, contents)
	layers = append(layers, layer)

	contents = getFileContents(OpenPolicyAgentPolicyFileName)
	layer = memoryStore.Add(OpenPolicyAgentPolicyFileName, OpenPolicyAgentPolicyLayerMediaType, contents)
	layers = append(layers, layer)

	contents = getFileContents(OpenPolicyAgentDataFileName)
	layer = memoryStore.Add(OpenPolicyAgentDataFileName, OpenPolicyAgentDataLayerMediaType, contents)
	layers = append(layers, layer)

	// Push
	fmt.Printf("Pushing OPA bundle to %s...\n", RemoteRef)
	extraOpts := []oras.PushOpt{oras.WithConfigMediaType(OpenPolicyAgentConfigMediaType)}
	manifest, err := oras.Push(context.Background(), resolver, RemoteRef, memoryStore, layers, extraOpts...)
	check(err)
	fmt.Printf("Pushed OPA bundle to %s with digest %s\n", RemoteRef, manifest.Digest)
}

and the resulting manifest:

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.cncf.openpolicyagent.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.cncf.openpolicyagent.manifest.layer.v1+json",
      "digest": "sha256:a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447",
      "size": 12,
      "annotations": {
        "org.opencontainers.image.title": ".manifest"
      }
    },
    {
      "mediaType": "application/vnd.cncf.openpolicyagent.rego.layer.v1+rego",
      "digest": "sha256:a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447",
      "size": 12,
      "annotations": {
        "org.opencontainers.image.title": "helpers.rego"
      }
    },
    {
      "mediaType": "application/vnd.cncf.openpolicyagent.policy.layer.v1+rego",
      "digest": "sha256:a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447",
      "size": 12,
      "annotations": {
        "org.opencontainers.image.title": "deny.rego"
      }
    },
    {
      "mediaType": "application/vnd.cncf.openpolicyagent.data.layer.v1+json",
      "digest": "sha256:a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447",
      "size": 12,
      "annotations": {
        "org.opencontainers.image.title": "samples.json"
      }
    }
  ]
}

Hope this helps! It's interesting to see this concept being brought to other projects.

@jdolitsky thanks, the extra type was a typo from a previous version. Fixed now.

Thanks for the code, I'll use that as the starting point. I have the pull side already implemented.

I've submitted the top level media type to the ACR registry Azure/acr#225. I'll close this issue when that gets merged. Thanks everyone.

I'm going to close this since everyone is on board. If you think there is anything we should do in OPA around this, feel free to file more issues. Cheers!

This topic needs to be brought to another forum, but there is probably a more appropriate long term place to store these “well known” media types (OCI?)
…
On Tue, May 14, 2019 at 1:33 AM Gareth Rushgrove @.***> wrote: I've submitted the top level media type to the ACR registry Azure/acr#225 <Azure/acr#225>. I'll close this issue when that gets merged. Thanks everyone. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1413?email_source=notifications&email_token=AADACFRF5YME7XDX6R6L6M3PVJMMPA5CNFSM4HLICY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVKNBXQ#issuecomment-492097758>, or mute the thread https://github.com/notifications/unsubscribe-auth/AADACFQVE6T57IHD4VYBJILPVJMMPANCNFSM4HLICY4A .

Just commenting on the location for mediaTypes.
As we continue to work through the OCI TOB, we're proposing having well known artifacts submit their mediaTypes for an artifact using this type of format: artifactTypes
A description for how to author these is here: Registering well known artifact types

Steve