Skip to content

Llamafile comparison? #8

Description

@hrstoyanov

@mukel thank you for creating this project! I would like to discuss the following topics:

  1. Please enable the Discussions tab for posts like this, which are not real "issues"

  2. Do you plan on releasing Llama3 code?

  3. Do you plan on quantized llama models with Java vector api?

  4. Can you run a benchmark against llamafile, the vector version of which (AVX, neon) claims to be the performance king for inference.
    (I am deciding between using your project or wrapping around the llamafile c code with Java 22 foreign function apis)

  5. Do you plan to implant model training as well? If so, take a look at Andrey's LLM.c repo

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions