typhoon-chat.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[],"dockerImageVersionId":30733,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"!pip install accelerate\n!pip install transformers\n!pip install torch","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-07-01T13:47:53.615639Z","iopub.execute_input":"2024-07-01T13:47:53.616514Z","iopub.status.idle":"2024-07-01T13:48:34.323809Z","shell.execute_reply.started":"2024-07-01T13:47:53.616469Z","shell.execute_reply":"2024-07-01T13:48:34.322659Z"},"trusted":true},"execution_count":1,"outputs":[{"name":"stdout","text":"Requirement already satisfied: accelerate in /opt/conda/lib/python3.10/site-packages (0.30.1)\nRequirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from accelerate) (1.26.4)\nRequirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from accelerate) (21.3)\nRequirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate) (5.9.3)\nRequirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from accelerate) (6.0.1)\nRequirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from accelerate) (2.1.2)\nRequirement already satisfied: huggingface-hub in /opt/conda/lib/python3.10/site-packages (from accelerate) (0.23.2)\nRequirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.10/site-packages (from accelerate) (0.4.3)\nRequirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->accelerate) (3.1.1)\nRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.13.1)\nRequirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (4.9.0)\nRequirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (1.12.1)\nRequirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.2.1)\nRequirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1.2)\nRequirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2024.3.1)\nRequirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2.32.3)\nRequirement already satisfied: tqdm>=4.42.1 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub->accelerate) (4.66.4)\nRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\nRequirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.3.2)\nRequirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.6)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (1.26.18)\nRequirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2024.2.2)\nRequirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\nRequirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.41.2)\nRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)\nRequirement already satisfied: huggingface-hub<1.0,>=0.23.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.23.2)\nRequirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.4)\nRequirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (21.3)\nRequirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)\nRequirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)\nRequirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.32.3)\nRequirement already satisfied: tokenizers<0.20,>=0.19 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.19.1)\nRequirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.3)\nRequirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.66.4)\nRequirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.0->transformers) (2024.3.1)\nRequirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.0->transformers) (4.9.0)\nRequirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->transformers) (3.1.1)\nRequirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.3.2)\nRequirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.6)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.18)\nRequirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2024.2.2)\nRequirement already satisfied: torch in /opt/conda/lib/python3.10/site-packages (2.1.2)\nRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch) (3.13.1)\nRequirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch) (4.9.0)\nRequirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch) (1.12.1)\nRequirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch) (3.2.1)\nRequirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch) (3.1.2)\nRequirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch) (2024.3.1)\nRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch) (2.1.3)\nRequirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/lib/python3.10/site-packages (from sympy->torch) (1.3.0)\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Import important lib and download LLM model ( Text generate )","metadata":{}},{"cell_type":"code","source":"import torch \nfrom transformers import AutoModelForCausalLM, AutoTokenizer, pipeline\n\nmodel_name = \"scb10x/llama-3-typhoon-v1.5x-8b-instruct\"\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    device_map=\"auto\",\n    torch_dtype=torch.float16,\n    trust_remote_code=True,\n)\n\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\n# Explicitly set pad_token_id to eos_token_id\ntokenizer.pad_token_id = tokenizer.eos_token_id\n\npipe = pipeline(\"text-generation\", model=model, tokenizer=tokenizer)\n\ngeneration_args = {\n    \"max_new_tokens\": 512,\n    \"return_full_text\": False,\n    \"pad_token_id\": tokenizer.pad_token_id, \n}","metadata":{"execution":{"iopub.status.busy":"2024-07-01T13:48:34.325782Z","iopub.execute_input":"2024-07-01T13:48:34.326085Z","iopub.status.idle":"2024-07-01T13:51:39.053883Z","shell.execute_reply.started":"2024-07-01T13:48:34.326057Z","shell.execute_reply":"2024-07-01T13:51:39.053155Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stderr","text":"2024-07-01 13:48:41.537008: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n2024-07-01 13:48:41.537160: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n2024-07-01 13:48:41.671245: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"config.json:   0%|          | 0.00/615 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b74b1a9dd7b24e1795c62cf5a5a122f6"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1735ac7d29fd43aba257091eef3eaa92"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading shards:   0%|          | 0/4 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e9d03225ebb647198d91aed5ca5605c2"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00001-of-00004.safetensors:   0%|          | 0.00/4.98G [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0129a4689d384f53bdd637a4b49eb24e"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00002-of-00004.safetensors:   0%|          | 0.00/5.00G [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"024665076065414f83b6185410bc452e"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00003-of-00004.safetensors:   0%|          | 0.00/4.92G [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bf02a8ec72924a9584832ba723bd654f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"model-00004-of-00004.safetensors:   0%|          | 0.00/1.17G [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ed2761c02f174ae2878d6f9651a0c046"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c55f4984256b4342bb39842ea398a355"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"generation_config.json:   0%|          | 0.00/121 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"25a5d66b99b64763a18e3fb70ebfb0e1"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"tokenizer_config.json:   0%|          | 0.00/51.0k [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"234e1e6253414c04a48491932b1849f4"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"tokenizer.json:   0%|          | 0.00/9.09M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"52197cfae62b4910bb57a1211ef6c745"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"special_tokens_map.json:   0%|          | 0.00/301 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4628b66c47df40bb91f66ff128630e8a"}},"metadata":{}},{"name":"stderr","text":"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Function Called LLM","metadata":{}},{"cell_type":"code","source":"def query(messages):\n\n  output = pipe(messages, **generation_args)\n  return output[0]['generated_text']","metadata":{"execution":{"iopub.status.busy":"2024-07-01T13:51:39.055114Z","iopub.execute_input":"2024-07-01T13:51:39.055893Z","iopub.status.idle":"2024-07-01T13:51:39.060861Z","shell.execute_reply.started":"2024-07-01T13:51:39.055859Z","shell.execute_reply":"2024-07-01T13:51:39.059819Z"},"trusted":true},"execution_count":3,"outputs":[]},{"cell_type":"markdown","source":"Chat modify function ( loop ) ( very high GPU and memory usage )","metadata":{}},{"cell_type":"code","source":"def chat():\n\n  conversation_history = [\n      {\"role\": \"system\", \"content\" : \"You are Dev God , you can plan for model business , you are very good at business ,you will give every single details about business that user ask\"\n      }]\n  while True:\n    user_input = input(\"You: \")\n    if user_input.lower() == \"exit\":\n        break\n\n    conversation_history.append({\"role\": \"user\", \"content\": user_input})\n\n    try:\n        response = query(conversation_history)\n        print(\"Teacher: \", response)\n\n        conversation_history.append({\"role\": \"assistant\", \"content\": response})\n\n    except Exception as e:\n        print(f\"An error occurred: {e}, please try again.\")","metadata":{"execution":{"iopub.status.busy":"2024-07-01T13:51:39.062706Z","iopub.execute_input":"2024-07-01T13:51:39.062979Z","iopub.status.idle":"2024-07-01T13:51:39.090590Z","shell.execute_reply.started":"2024-07-01T13:51:39.062954Z","shell.execute_reply":"2024-07-01T13:51:39.089630Z"},"trusted":true},"execution_count":4,"outputs":[]},{"cell_type":"code","source":"chat()","metadata":{"execution":{"iopub.status.busy":"2024-07-01T13:51:39.092933Z","iopub.execute_input":"2024-07-01T13:51:39.093415Z"},"trusted":true},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdin","text":"You:  do you know web3 ?\n"},{"name":"stdout","text":"Teacher:  Yes, I am familiar with Web3. Web3 is the third generation of the World Wide Web, which focuses on decentralization, security, and privacy. It is built on blockchain technology and aims to create a more decentralized and open internet.\n\nWeb3 is an evolution of the traditional web, which is centralized and controlled by a few large corporations. Web3 aims to give users more control over their data and enable them to interact with the internet in a more secure and private way.\n\nWeb3 is still in its early stages of development, but it has the potential to revolutionize the way we interact with the internet and the way we do business. Some of the key features of Web3 include:\n\n1. Decentralization: Web3 is built on blockchain technology, which allows for decentralized and trustless transactions.\n2. Security: Web3 is more secure than the traditional web because it uses cryptography and blockchain technology to protect user data.\n3. Privacy: Web3 enables users to maintain their privacy and control over their data.\n4. Interoperability: Web3 allows for the integration of different blockchain networks and protocols, enabling seamless communication and collaboration between them.\n\nWeb3 has the potential to create new business models and opportunities, such as decentralized finance (DeFi), non-fungible tokens (NFTs), and decentralized applications (dApps). However, it also presents challenges and risks, such as regulatory uncertainty, security vulnerabilities, and the potential for scams and fraud.\n\nAs a business strategist, I can help you understand the opportunities and challenges of Web3 and develop a strategy for leveraging its potential in your business.\n","output_type":"stream"},{"output_type":"stream","name":"stdin","text":"You:  any Business model for Travel that using Web 3\n"},{"name":"stdout","text":"Teacher:  Yes, there are several business models for the travel industry that can leverage Web3 technology. Here are a few examples:\n\n1. Decentralized Travel Booking Platform: A decentralized travel booking platform can use blockchain technology to create a transparent and secure platform for travelers to book flights, hotels, and other travel services. The platform can use smart contracts to ensure that transactions are secure and that travelers receive the services they paid for.\n2. Travel Rewards Program: A travel rewards program can use Web3 technology to create a decentralized rewards system that allows travelers to earn and redeem rewards in a secure and transparent way. The rewards can be in the form of tokens or other digital assets that can be used to book travel services or redeemed for other rewards.\n3. Decentralized Travel Marketplace: A decentralized travel marketplace can use Web3 technology to create a platform that connects travelers with local travel providers, such as tour guides, transportation services, and accommodation providers. The platform can use smart contracts to ensure that transactions are secure and that travelers receive the services they paid for.\n4. Travel Insurance: A travel insurance company can use Web3 technology to create a decentralized insurance platform that allows travelers to purchase insurance policies in a secure and transparent way. The platform can use smart contracts to ensure that insurance claims are processed quickly and fairly.\n5. Decentralized Travel Community: A decentralized travel community can use Web3 technology to create a platform that connects travelers with other travelers and travel providers. The platform can use social media and other tools to enable travelers to share their experiences, recommendations, and reviews with others.\n\nThese are just a few examples of the many business models that can leverage Web3 technology in the travel industry. As a business strategist, I can help you develop a business model that leverages Web3 technology and meets the needs of your target market.\n","output_type":"stream"},{"output_type":"stream","name":"stdin","text":"You:  any new idea to make my travel business very unique\n"},{"name":"stdout","text":"Teacher:  Yes, here are a few ideas that can help make your travel business unique:\n\n1. Personalized Travel Experiences: Use Web3 technology to create a platform that allows travelers to customize their travel experiences based on their preferences, interests, and budget. The platform can use machine learning algorithms to suggest personalized itineraries, activities, and accommodations.\n2. Community-Based Travel: Create a platform that connects travelers with local communities and enables them to participate in local events, activities, and cultural experiences. This can help travelers gain a deeper understanding of the local culture and contribute to the local economy.\n3. Sustainable Travel: Focus on sustainable travel practices and create a platform that encourages travelers to reduce their carbon footprint. The platform can provide information on eco-friendly accommodations, transportation, and activities, and offer incentives for travelers who choose sustainable options.\n4. Virtual Travel Experiences: Create a virtual reality platform that allows travelers to explore destinations and experiences remotely. This can be especially useful for travelers who are unable to travel physically, or for those who want to explore a destination before booking a trip.\n5. Travel Education: Create a platform that provides educational resources and training for travelers, such as language courses, cultural immersion programs, and travel safety tips. This can help travelers become more informed and prepared for their trips.\n6. Travel Rewards Program: Create a decentralized travel rewards program that allows travelers to earn and redeem rewards in a secure and transparent way. The rewards can be in the form of tokens or other digital assets that can be used to book travel services or redeemed for other rewards.\n7. Travel Insurance: Create a decentralized travel insurance platform that allows travelers to purchase insurance policies in a secure and transparent way. The platform can use smart contracts to ensure that insurance claims are processed quickly and fairly.\n\nThese are just a few ideas that can help make your travel business unique. As a business strategist, I can help you develop a unique value proposition and marketing strategy that sets your business apart from the competition.\n","output_type":"stream"},{"name":"stderr","text":"\nKeyboardInterrupt\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"@ i wish my book will help you to get start about llm easier [github](https://github.com/idkbreh)","metadata":{}}]}