Add Gen-AI instructions to Coding Guidelines

[dlmarion]

No description provided.

[dlmarion]

I looked at some commit message in Apache repositories to see how other projects are handling gen-ai attribution in commit messages. Here is one from Thrift, Hadoop and Airflow.

My suggestion to not use Co-Authored-By as that format should include an email address and maybe we save that attribution for humans.

[ddanielr]

I agree with the Assisted-By or Generated-by commit message format and leaving Co-Authored-By for human authors.

If we wanted to standardize on a single one then I would vote for using Assisted-By.

[phrocker]

I think https://www.apache.org/legal/generative-tooling.html#include-in-contributions suggests Generated-by: for provenance.

Assisted-by: leans toward claiming more human authorship while disclosing less provenance. Generated-by: makes no authorship claim at all, it just names a tool, which is the cheaper and more honest thing to assert.

[dlmarion]

I think https://www.apache.org/legal/generative-tooling.html#include-in-contributions suggests Generated-by: for provenance.

Assisted-by: leans toward claiming more human authorship while disclosing less provenance. Generated-by: makes no authorship claim at all, it just names a tool, which is the cheaper and more honest thing to assert.

I suggested two options because I think there is a distinction. To me Generated-By implies that the machine did all the work. This could be true in some cases where the developer uses an AI harness-only to iterate and modify code and the human just does review. But, in the case where the developer only uses a machine to do part of the work Generated-By implies that the developer did little to no work.

Maybe I'm taking it too personally :-).

I re-read the paragraph that uses Generated-By as an example and if the requirement is for provenance and traceability rather than some type of assignment of authorship, then maybe a more generic tag can be used, say AI-Model-Used:.

[DomGarguilo]

To me Generated-By implies that the machine did all the work. This could be true in some cases where the developer uses an AI harness-only to iterate and modify code and the human just does review. But, in the case where the developer only uses a machine to do part of the work Generated-By implies that the developer did little to no work.

This is how Generated-By vs Assisted-By reads in my head too. In terms of what is most correct/best practice here I am not sure.

If we wanted to standardize on a single one then I would vote for using Assisted-By.

This suggestion sounds reasonable to me too

[ddanielr]

I suggested two options because I think there is a distinction. To me Generated-By implies that the machine did all the work. This could be true in some cases where the developer uses an AI harness-only to iterate and modify code and the human just does review. But, in the case where the developer only uses a machine to do part of the work Generated-By implies that the developer did little to no work.

I was also thinking that Generated-by has connotations of minimal human involvement.

I think https://www.apache.org/legal/generative-tooling.html#include-in-contributions suggests Generated-by: for provenance.

Assisted-by: leans toward claiming more human authorship while disclosing less provenance. Generated-by: makes no authorship claim at all, it just names a tool, which is the cheaper and more honest thing to assert.

I see your point about Assisted-by. I'm mostly thinking of this wording in the context of PR reviews.
In a perfect environment all reviews are getting the same level of testing and review but that's generally not the case.

Personally, It would be helpful to know which files/commits were completely generated code so I know what areas of the code review to focus on.

As an example, we have thrift generate our rpc classes and generally that code is not reviewed. Instead we focus on the behavior tests passing.

The only specific check that does occur ensures the generated content has not changed unintentionally.

I accept that I might be thinking about this too much.
If we don't want to be that granular then maybe AI-Model-Used: is a good way of bridging that gap?

[phrocker]

I suggested two options because I think there is a distinction. To me Generated-By implies that the machine did all the work. This could be true in some cases where the developer uses an AI harness-only to iterate and modify code and the human just does review. But, in the case where the developer only uses a machine to do part of the work Generated-By implies that the developer did little to no work.

I was also thinking that Generated-by has connotations of minimal human involvement.

I think https://www.apache.org/legal/generative-tooling.html#include-in-contributions suggests Generated-by: for provenance.
Assisted-by: leans toward claiming more human authorship while disclosing less provenance. Generated-by: makes no authorship claim at all, it just names a tool, which is the cheaper and more honest thing to assert.

I see your point about Assisted-by. I'm mostly thinking of this wording in the context of PR reviews. In a perfect environment all reviews are getting the same level of testing and review but that's generally not the case.

Personally, It would be helpful to know which files/commits were completely generated code so I know what areas of the code review to focus on.

As an example, we have thrift generate our rpc classes and generally that code is not reviewed. Instead we focus on the behavior tests passing.

The only specific check that does occur ensures the generated content has not changed unintentionally.

I accept that I might be thinking about this too much. If we don't want to be that granular then maybe AI-Model-Used: is a good way of bridging that gap?

I wouldn't say I would have very much of a strong reaction to Assisted-By or Generated-By, but AI-Model-Used avoids any reference to how the code was generated and it's quite binary about its intent, which I like. It's a good discussion point to see if anyone else disagrees, but I like AI-Model-Used for its simplicity.

[ctubbsii]

The ASF guidelines are still evolving, and are currently in draft. While the legal page is binding, and shows an example, there are additional draft best practices at https://rai.apache.org/best-practices.html

Much of this is currently redundant, and conflicting. I have read it over and provided feedback to the RAI committee. My personal recommendation is that we should use Co-Authored-By: <model>. While the standard format of the Author and Committer fields include an email address, it is actually optional and not strictly required by git. The "Co-Authored-By" tag inside the body of the commit follows the same format, so it is also optional to include an email address, and I would just omit them, since there isn't an email for an AI model.

The RAI best practices draft pages also document other alternatives, which I think are less good. I don't believe "AI-Model-Used" is one of them, and I would strongly advise that we not invent something new. I think sticking to "Co-Authored-By" is the most obvious and already understood by existing tooling. It also covers many other cases (assisted-by, generated-by, generated-then-human-reviewed, reviewed-by, etc.), because nobody expects this tag to communicate the specific quantity and nature of each of the contributions of each author, and nobody is asking for that because that'd be overkill. Let's keep it simple, and jut use Co-Authored-By: <model>

[dlmarion]

I have no issue changing AI-Model-Used to something else. Do we have an idea of when the ASF guidelines will come out of draft? Looking at https://rai.apache.org/authored-by-tags.html, the draft is recommending ai-assisted-by and possibly Apache-ai tags.

[ctubbsii]

No idea when it will be final. But in discussion with one of the RAI members, they said they were going to update them soon, and that I would probably be happier with the changes (after I provided feedback similar to here). Even after they are final, it will still just be recommendations, not requirements. So, we can do whatever we want. The most important thing is just marking that it was used by some mechanism. The "how" doesn't matter except for personal/group preference. For our contributor guidelines, I'd keep things very minimal... just a link to ASF legal policy, a possible link to the RAI draft guidelines (just the top page, since the pages might change) with a warning that they are subject to change, and maybe a general suggestion to use "Co-Authored-By: or a similar acknowledgment in the commit message, with information about the AI model used", rather than dictate a specific format.

[ctubbsii]

-1 to these changes. (See my earlier comment)

[ctubbsii]

It's fine as-is. Could make this one change, but it's fine either way: