Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: hide gpu check windows #993

Merged
merged 1 commit into from
Aug 7, 2024
Merged

fix: hide gpu check windows #993

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions cortex-js/src/usecases/models/models.usecases.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,9 @@ import { TelemetrySource } from '@/domain/telemetry/telemetry.interface';
import { ModelRepository } from '@/domain/repositories/model.interface';
import { ModelParameterParser } from '@/utils/model-parameter.parser';
import {
HuggingFaceRepoData,
HuggingFaceRepoSibling,
} from '@/domain/models/huggingface.interface';
import {
fetchHuggingFaceRepoData,
fetchJanRepoData,
getHFModelMetadata,
} from '@/utils/huggingface';
Expand Down Expand Up @@ -530,16 +528,6 @@ export class ModelsUsecases {
else throw 'Model already exists.';
}

/**
* Fetches the model data from HuggingFace
* @param modelId Model repo id. e.g. llama3, llama3:8b, janhq/llama3
* @returns Model metadata
*/
fetchModelMetadata(modelId: string): Promise<HuggingFaceRepoData> {
if (modelId.includes('/')) return fetchHuggingFaceRepoData(modelId);
else return fetchJanRepoData(modelId);
}

/**
* Get the current status of the models
* @returns Model statuses
Expand Down
3 changes: 3 additions & 0 deletions cortex-js/src/utils/cuda.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@ export const getGpuInfo = async (): Promise<GpuSettingInfo[]> =>
new Promise((resolve) => {
exec(
'nvidia-smi --query-gpu=index,memory.total,name --format=csv,noheader,nounits',
{
windowsHide: true,
},
async (error, stdout) => {
if (!error) {
// Get GPU info and gpu has higher memory first
Expand Down
46 changes: 0 additions & 46 deletions cortex-js/src/utils/huggingface.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,52 +60,6 @@ export function guessPromptTemplateFromHuggingFace(jinjaCode?: string): string {
}
}

/**
* Fetches the model data from HuggingFace API
* @param repoId HuggingFace model id. e.g. "janhq/llama-3"
* @returns
*/
export async function fetchHuggingFaceRepoData(
repoId: string,
): Promise<HuggingFaceRepoData> {
const sanitizedUrl = getRepoModelsUrl(repoId);

const { data: response } = await axios.get(sanitizedUrl);
if (response['error'] != null) {
throw new Error(response['error']);
}

const data = response as HuggingFaceRepoData;

if (data.tags.indexOf('gguf') === -1) {
throw `${repoId} is not supported. Only GGUF models are supported.`;
}

// fetching file sizes
const url = new URL(sanitizedUrl);
const paths = url.pathname.split('/').filter((e) => e.trim().length > 0);

for (let i = 0; i < data.siblings.length; i++) {
const downloadUrl = HUGGING_FACE_DOWNLOAD_FILE_MAIN_URL(
[paths[2], paths[3]].join('/'),
data.siblings[i].rfilename,
);
data.siblings[i].downloadUrl = downloadUrl;
}

//TODO: Very hacky? Let's say they don't name it properly
AllQuantizations.forEach((quantization) => {
data.siblings.forEach((sibling: any) => {
if (!sibling.quantization && sibling.rfilename.includes(quantization)) {
sibling.quantization = quantization;
}
});
});

data.modelUrl = HUGGING_FACE_REPO_URL(paths[2], paths[3]);
return data;
}

/**
* Fetch the model data from Jan's repo
* @param modelId HuggingFace model id. e.g. "llama-3:7b"
Expand Down
Loading