+1 binding
I would like to bring the end-user perspective to the discussion. At this stage, there are different requirements to operate a k8s cluster and some teams and companies are using k3s (due to its opinionated structure) as a solid exploration point. It benefits the community as a project and I CNCF will be a great home to further the growth of this project.
Katie Gamanji
The chasm statement is interesting in the abstract. The well-deserved implication is “production use” is far to broad a term for a huge variety of situations, and I withdraw the question. Has the project made it clear how and when security
issues (both under embargo and not) that are upstream will be handled, or not? That way the users can make their own judgements about whether that is acceptable risk for whatever their usage scenario is.
Conformance is a set of test cases. You don’t need to (nor should you) have to have any set of upstream code to pass conformance. This is good and as it should be. It can be both true that a project is conformant, and be either a fork or
a complete re implementation. We tripped over this way back when during the Openstack days, and I think the Kubernetes project took the lesson. It is also possible to configure a non-forked/upstream distro to not pass conformance.
Consequently, statements of the form, “x is not a fork because it is conformant” are misleading.
Forks represent a compatibility risk for users. Again, the users are the ones that need to evaluate the risk and decide how fully they believe conformance covers their intended usage. We should be clear with the communities and users so
they are able to make their own judgements.
To your specific comment: It’s not about whether etcd is present. That’s not what I was saying. It’s about whether the code that is needed to replace etcd with something else is a fork.
-bw
From: Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 12:21 PM
To: "Wise, Bob" <wisebob@...>, CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox
|
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless
you can confirm the sender and know the content is safe.
|
I can speak to some of this...
Are we clear about sandbox projects as “not recommended for production”
The CNCF describes projects in terms of crossing the chasm. You can find that
here. Sandbox is labeled as being for the innovators.
A production recommendation really depends on who you are. An enterprise is different from an early stage startup. What one would suggest be used in prod for a new startup or for The Home Depot is going to be very different. I like the
description in terms of the technology adoption lifecycle instead.
The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
This brings up a good question that, I think, is a bit more complicated. What parts of Kubernetes are required to be present to be conformant? I would leave this up to the conformance folks. For example, would the use of Virtual Kubelet
(another sandbox project) be a change to mean to far to allow? Why?
From the perspective of someone deploying workloads into a cluster I don't see how it matters if the API and environment is conformant for me to run workloads in.
If I'm missing something I would be curious to know.
For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.
This idea isn't new. It has been brought up before. While I can't state where this sits today I do know that bringing in new ideas to k8s can be difficult now because of people working in certain ways to maintain them. In the past SIG Architecture
would tell people to innovate things out of tree. If it was a good idea and proved to be useful then K8s would potentially look to bring it back in house.
I like this idea. But, I think it needs to be proved out of tree before it can be brought in.
On Mon, Aug 10, 2020, at 1:48 PM, Wise, Bob wrote:
To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:
-
The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
-
The precedent setting for both distros and forks.
-
Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended
for production”?
Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?
For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.
-Bob
From:
<cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox
|
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm
the sender and know the content is safe.
|
I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.
There are two things about this that jump out to me.
First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.
kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around
a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."
This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.
k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using
it to look different.
It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.
Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself
in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.
The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly
tell them to do them as part a different project rather than in k8s.
I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.
I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.
This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable
insight for a CNCF application.
I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one
distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.
The sandbox is a great please to experiment in a cross company way.
The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects
of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.
Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.
Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.
On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:
Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.
Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.
I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.
This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable
insight for a CNCF application.
While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.
Why not vote "-1"? The
revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation.
This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.
|
