Hosting an LLM on a local device:

Context / Prerequisites:  

The local device used is a Windows 10 Local PC. The remote software used to capture images is Any Desk.

Website link here: https://anydesk.com/en

Navigate to LM Studio and download the LM Studio application.  (Version 0.323 Build 3)

LM Studio link here: lmstudio.ai 

Complete the download wizard and select finish.

Since this is the first time setup I’ll be selecting the User option

After installing LM Studio we need to install a language model according to my use case. For this instance I will be installing openai/gpt-oss-20b. It fits my device component requirements. It also has access to a large pool of data and holds its own with writing analysis and generation.

Image and data pulled from:
“Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI’s Latest Open Source Models” LINK HERE

Here we select our First LLM to download. Select the download button and wait for it to finish.

=== This information is not required but can help with selecting models for future projects ===

Test information and links attached for additional context/documentation on tests provided below:

FINQA: Numerical Reasoning of Financial Data

PIQA: Physical Commonsense Reasoning

CEVAL: Evaluating Counterfactual Text Generation

DIALOGSUM: A Real-Life Scenario Dialogue Summarization Dataset

LEGALQA: Legal Understanding

GSM8K: Training Verifiers to Solve Math Word Problems

SCIQ: Scientific Reasoning

MEDQA: Medical Knowledge Assessment

MMLU: Measuring Massive Multitask Language Understanding

Once the LLM has downloaded, be sure to ask it questions and verify that it is working as intended.

This concludes the primary goal of this project.

You can continue reading to see how you would install different models provided by LM Studio.

Next Search the desired Model in the search bar and select Download in the bottom right.