Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract/virtual methods for extern objects #561

Closed
9 tasks done
mihaibudiu opened this issue Jan 24, 2018 · 15 comments
Closed
9 tasks done

Abstract/virtual methods for extern objects #561

mihaibudiu opened this issue Jan 24, 2018 · 15 comments

Comments

@mihaibudiu
Copy link
Contributor

mihaibudiu commented Jan 24, 2018

Personnel

Design

  • Document: see below for a short description

Implementation

Process

  • abstract methods in extern objects; "this" reference.

This allows an extern to declare that a method has to be implemented by a user in P4. The syntax in the extern declaration is:

extern E {
   E();
   abstract bit<32> toImplement(in bit<32> data);
}

When the extern is instantiated the user must implement all the abstract methods:

E() instance = {
    bit<32> toImplement(in bit<32> data) { return data + 1; }
};

Inside such a method implementation one can use the "this" pointer to refer to the instance, e.g., by invoking other methods of that object.

This extern is really a black box that has inside a "white box" which is a P4 program. This is useful for extern accelerators that have user-configurable policies that can be expressed in P4. The "packet transactions" paper from SIGCOMM '16 shows several use cases.

@antoninbas
Copy link
Member

Regarding abstract method, please consider supporting by-reference capture of variables in scope when implementing the method. See #446 (comment) for a use-case.

@mihaibudiu
Copy link
Contributor Author

That won't be possible, because

  1. P4 does not have references
  2. It is not clear when these methods are evaluated, so they could cause side-effects at any time during the P4 program evaluation. An extern method can be invoked at any time, it does not need to be invoked by the data-plane synchronously.

We can't even allow read-only capture, because we don't control when the value of the variables is being read.

@jafingerhut
Copy link
Contributor

I'm sure there are complexities I'm not considering here, but just a basic question from the peanut gallery:

In the particular example that Antonin links to, with read-only capture, at least in that particular case the time of the read-only capture seems pretty clearly desired to be the time that the table with the ActionSelector extern is apply'd.

Perhaps one of the issues for the compiler that you are concerned with, Mihai, is that there might become multiple extern's defined with different rules for determining its capture time?

@mihaibudiu
Copy link
Contributor Author

There is no capture time since all externs are defined at compile-time.
There is the evaluation time. We don't know when the abstract method will be evaluated. In general, no one will call that method, it is called by the extern internally at an unspecified time. It could even be called because the control-plane asks for it.

@jafingerhut
Copy link
Contributor

P4_16 PSA externs regularly define methods that are only allowed to be called from the data plane, never the control plane. The proposed example appears to be one such case.

Evaluation time is clearly meant to be at table apply() call time of the table with that action selector as its table property value. Whether that is true of all possible use cases someone might imagine for such a feature remains to be seen from people's imagination, but if those imaginations were restricted by "you must clearly define when the evaluation time is for such functions", does that simplify things enough?

To my knowledge, every compiler back-end must know about the details of every extern they implement. For example, I expect back end compilers for PSA will freely allow reordering of counter update calls, but they can only do this because the compiler writer knows that this is safe to do. If they didn't know what a counter's behavior was, such reordering would be incorrect.

@mihaibudiu
Copy link
Contributor Author

Abstract methods were built for a use case where they are not necessarily called from the p4 program directly.

P4 is defined is such a way that the effect of an extern method call on the dataplane program is limited to modify only parameters that are out and to only read parameters that are in, no matter what the extern methods are doing. Moreover, the P4 design assumes that all calls to extern methods that can influence the dataplane appear explicitly in the program. This does not hold for abstract methods.

@milad14000
Copy link

@mbudiu-vmw The abstract function implementation in Antonin's example is very similar to how we define actions in p4. The following action is a valid p4, and technically meta.hash2 is passed by reference and we only read its value when the table is hit.

action drop() {
    smeta.drop = 1;
    meta.hash1 = meta.hash2
}

But I do agree with your point that an extern method can be invoked at any time and it is not clear when the value of the variables is being read.

@mihaibudiu
Copy link
Contributor Author

meta.hash2 is not passed by reference, it is just in scope, so it can be used directly.
But for actions we know exactly when they may be executed: either when they are called directly, or when the table containing them is invoked. A table is just a nondeterministic switch statement which executes one of the actions in the list of actions. So we know that a table call may modify the variables that are in scope when it is executed.

An abstract function can be called once or multiple times or never by the extern when some other extern method is invoked, or even possibly when a control-plane API of the extern is invoked.

@mihaibudiu mihaibudiu changed the title Experimental features in P4 compilers Abstract/virtual methods for extern objects Nov 30, 2018
@mihaibudiu
Copy link
Contributor Author

I have renamed this issue, since all of the other experimental features that were listed have been solved.
Most of the discussion is only about abstract methods.

@jnfoster
Copy link
Contributor

jnfoster commented Jan 7, 2021

This experimental feature is used to model RegisterActions in Barefoot's Open-Tofino TNA architecture.

https://github.com/barefootnetworks/Open-Tofino/blob/master/share/p4c/p4include/tofino.p4#L659

@jafingerhut
Copy link
Contributor

jafingerhut commented Jan 10, 2021

@mbudiu-vmw You wrote earlier: "An abstract function can be called once or multiple times or never by the extern when some other extern method is invoked, or even possibly when a control-plane API of the extern is invoked."

Imagine an abstract method that by its definition can read and/or write variables in a P4 program, via direction in, out, or inout parameters.

Suppose a data plane is currently processing NO packets at all, because there are none available to process.

What would it even mean for a control-plane API of the extern being invoked, to cause the abstract function to be called? There are NO P4 variables to read or write at that time, at all, are there?

If we agree that scenario makes no sense, then there are at least two ways to interpret that:

(a) abstract methods with in, out, or inout parameters make no sense, and should not be part of the P4 language. This might be what you are arguing for?

(b) It makes no sense for abstract methods with in, out, or inout parameters to be invoked except while a packet is being processed. Perhaps it should even be restricted to being invoked as a result of the current packet being processed only, not for some other packet. That is, it should be impossible for a packet A to somehow cause an abstract method to be executed, and read or write the values of P4 variables of some other packet B being processed.

For the RegisterAction in Barefoot's Open-Tofino TNA architecture mentioned in the previous comment, here: https://github.com/barefootnetworks/Open-Tofino/blob/master/share/p4c/p4include/tofino.p4#L659

I believe the intent is that the abstract method named apply can only be executed as a result of the method execute being called in a P4 program. The P4 program invokes the execute method, and as a result the architecture reads some state inside of the Register extern, then causes the apply method to be executed, and when apply finishes, only then can the execute method finish and return. The apply method will never be invoked at any other time, ever, guaranteed 100% by the architecture.

If the @synchronous annotation, or something similar, could be designed in such a way to indicate this restriction on when the abstract method can possibly be executed, to the P4 compiler, and no abstract method that read or wrote variables in its lexical environment could be executed at any other time, would that address your concerns?

If we don't address that question of restricting the execution time of abstract methods, and for which packet the abstract method is executed, I cannot see how this issue can be resolved in any way except "abstract methods don't belong in P4".

@mihaibudiu
Copy link
Contributor Author

Imagine an abstract method that by its definition can read and/or write variables in a P4 program, via direction in, out, or inout parameters.

I propose adding abstract methods in two stages.
In the first stage we would not allow abstract method implementations to touch any variables in scope. This indeed takes care of this objection.

In a later stage we would differentiate between "synchronous" and "asynchronous" methods. The synchronous methods can only be invoked as a result of a packet flowing through the pipeline; these methods would be allowed to access variables in scope. There are some additional constraints needed to make these accesses safe, but let's postpone that discussion for later.

Regarding parameters to abstract methods, these do make sense for both synchronous and asynchronous methods.

@vgurevich
Copy link

@ChrisDodd can comment, but I think that today's usage already uses the ability to access variables in scope and quite extensively. Having said that, I would like to be able to choose between doing that and passing these as parameters (like what we do in actions) -- depending on the usage one method is better than the other.

@mihaibudiu
Copy link
Contributor Author

yes, I know that the usage allows variable in scope. this is just about adding these two changes to the spec incrementally. think of it as breaking it into two patches.

@mihaibudiu
Copy link
Contributor Author

This was merged as part of #771

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants