Files
error-prone-support/openai-coder
2023-12-17 17:54:32 +01:00
..
2023-12-17 17:54:32 +01:00
2023-12-17 16:15:05 +01:00
2023-12-17 16:15:05 +01:00

OpenAI Coder

A command line utility that queries OpenAI for code fixes based on error and warning messages extracted from Maven build output.

Open topics and ideas.

  • Apply suggstions from https://clig.dev/.

  • Review ideas from http://catb.org/~esr/writings/taoup/html/ch10s05.html#id2948149 (e.g. -b).

  • Add support for an interactive mode, in which the user is prompted to confirm an identified issue before sending it to OpenAI.

    • In case multiple issues are reported against a single file, the user should be able to (a) select which issue(s) to send to OpenAI and (b) whether to batch the requests.
    • The user should be able to accept or reject the OpenAI response.
    • The user should be able to (re)try with a higher or lower temperature.
    • If a change is accepted, the user may opt to resubmit the modified file with a (new) subset of the reported issues.
    • The user should be able to modify the OpenAI request payload before sending it.

    (Check whether this can be done with Picocli (see below); otherwise check out JLine.)

  • Integrate with Picocli (or similar) to make the command line interface more user-friendly.

    • Add a --dry-run flag that only prints the issues that would be reported to OpenAI.
    • Add a --verbose flag that prints the OpenAI request and response payloads, or some abstraction thereof.
    • Add a --no-interactive flag that does not prompt the user to confirm the issue before sending it to OpenAI.
    • Add flags to include/exclude certain types of issues and files from being processed.
    • Add a --help flag that prints a help message.
    • Add a --run-to-fix <command> (name TBD) flag that repeatedly runs the given command in a sub-process and processes the output until either no further issues are reported, or no further fixes are found. (For this we could use ProcessBuilder.)
    • Add a --git-safe (name TBD) flag that only processes files that are tracked by Git and that have not been modified.
    • Add a --format <format> flag that allows the user to specify the format of the input. E.g. --format maven (default), --format errorprone, --format sarif, etc.
    • Add a --patch-context <lines> flag that allows the user to specify the number of lines of context to include in presented unified patches.
  • Create a binary image using GraalVM.

  • Add support for sending a suitable subset of the code to OpenAI, so as (a) to better deal with the token limit and (b) potentially reduce cost. This might take the form of using the line numbers of the issue to extract code section, as well as any relevant context. E.g. by replacing the source of irrelevant members with a unique placeholder. When the response is received, the placeholders are replaced with the original code.

  • Add support for parsing other formats (next to Maven build output). E.g. Sarif.

  • Add support for a mode in which a file (or if we can pull it off: a group of files) is explicitly specified to be processed using instructions entered interactively or non-interactively.

  • Introduce an IssueExtractor for Error Prone test compiler output.

  • Write an IssueExtractor for the output of the dependency:analyze Maven goal:

    [INFO] --- dependency:3.5.0:analyze-only (analyze-dependencies) @ openai-coder ---
    [WARNING] Used undeclared dependencies found:
    [WARNING]    com.google.code.findbugs:jsr305:jar:3.0.2:compile
    [WARNING]    com.theokanning.openai-gpt3-java:api:jar:0.12.0:compile
    [WARNING]    org.junit.jupiter:junit-jupiter-api:jar:5.9.2:test
    [WARNING]    io.github.java-diff-utils:java-diff-utils:jar:4.0:compile
    [WARNING] Unused declared dependencies found:
    [WARNING]    com.fasterxml.jackson.core:jackson-databind:jar:2.14.2:compile
    [WARNING]    tech.picnic.error-prone-support:refaster-runner:jar:0.9.1-SNAPSHOT:compile
    [WARNING]    com.google.errorprone:error_prone_check_api:jar:2.18.0:compile
    [WARNING]    com.google.errorprone:error_prone_annotation:jar:2.18.0:compile
    [WARNING]    com.google.errorprone:error_prone_test_helpers:jar:2.18.0:compile
    [WARNING]    org.springframework:spring-core:jar:5.3.26:compile
    

    This could e.g. be transformed to:

    [INFO] --- dependency:3.5.0:analyze-only (analyze-dependencies) @ openai-coder ---
    [WARNING] Used undeclared dependencies found:
    [WARNING]    com.google.code.findbugs:jsr305:jar:3.0.2:compile
    [WARNING]    com.theokanning.openai-gpt3-java:api:jar:0.12.0:compile
    [WARNING]    org.junit.jupiter:junit-jupiter-api:jar:5.9.2:test
    [WARNING]    io.github.java-diff-utils:java-diff-utils:jar:4.0:compile
    [WARNING] Unused declared dependencies found:
    [WARNING]    com.fasterxml.jackson.core:jackson-databind:jar:2.14.2:compile
    [WARNING]    tech.picnic.error-prone-support:refaster-runner:jar:0.9.1-SNAPSHOT:compile
    [WARNING]    com.google.errorprone:error_prone_check_api:jar:2.18.0:compile
    [WARNING]    com.google.errorprone:error_prone_annotation:jar:2.18.0:compile
    [WARNING]    com.google.errorprone:error_prone_test_helpers:jar:2.18.0:compile
    [WARNING]    org.springframework:spring-core:jar:5.3.26:compile
    

    (Though type and version can then conditionally be omitted.)

    The problem: the relevant pom.xml file is not available in the Maven output. It could be inferred from the preceding dependency:analyze line, but that line is logged at the INFO level, which is not extracted by the LogLineExtractor.

Interactive mode API ideation

For each file, the user is presented with a list of issues:

Issues in `/path/to/File.java`:
1. Issue 1.
2. Issue 2.
3. Issue 3.

Action [submit]: <prompt>

Upon submitting the issues to OpenAI, the user is presented with a unified patch of the suggested changes:

Suggested changes:
<diff>
 
Action [apply]: <prompt>

Commands:

  • If empty, then apply the appropriate default, depending on context:
    • If the most recent prompt listed the issues, then submit all issues to OpenAI (submit).
    • If the most recent prompt listed the suggested fixes, then apply the suggested fixes (apply).
  • h/help: Print help message.
  • i/issues: Print list of issues.
  • s/submit [<issue>, ...]: Submit all or the given issues to OpenAI.
  • a/apply: Apply the suggested changes:
    • If no changes have yet been suggested, then print a message and prompt again.
    • If changes have been suggested, then apply them, print a message and move on to the next file.
  • q/quit: Quit the program.
  • n/next: Skip the current file.
  • p/prev/previous: Go back to the previous file.