llama3.cuda

logits

  • 프롬프트: I have a dream [1, 76, 505, 263, 12561] 5개
  • 출력 (id와 logit값, 46개 출력으로 1개 더 많은 상태, CUDA)
    [29889:13.076488] [940:12.449268] [12561:11.759573] [29879:15.145534] [310:13.017419] [263:8.233423] [4802:4.261223] [29892:4.124094] [9560:5.366050] [16423:5.078463] [2989:12.229407] [310:23.905329] [18281:7.510609] [322:14.829545] [10697:8.289629] [29889:15.419161] [940:14.181862] [12561:11.183266] [29879:17.269192] [310:10.015463] [8743:6.805388] [411:12.010337] [670:10.454911] [7875:9.869209] [322:14.330401] [321:9.460655] [1218:19.804911] [343:10.416383] [11770:22.203684] [5807:10.086064] [26514:23.825844] [29889:16.950533] [13:14.384112] [6716:19.083275] [2462:12.955539] [29892:11.870152] [540:11.442266] [471:12.093337] [22049:10.019634] [297:13.978985] [278:14.038999] [16423:13.947722] [746:13.010663] [540:15.065728] [4446:14.374954] [263:15.728765]
    
  • 출력 (Python)
    [29889:13.105929362038037] [940:12.73067625436493] [12561:11.775138878217948] [29879:15.78920076722125] [310:12.843173873957994] [263:8.057945544400717] [4802:4.258677956772781] [29892:4.2125699986267735] [9560:5.341440519376919] [16423:5.154461198171746] [2989:11.990946787141965] [310:24.26916334946948] [18281:7.708171099718661] [322:14.909267287856556] [10697:8.18330034664377] [29889:15.33499487649984] [940:14.192116772451643] [12561:11.336398008635035] [29879:17.575418974756182] [310:10.190558156131136] [8743:6.872071118542993] [411:11.979582505929448] [670:10.62701438114209] [7875:9.677593240500208] [322:14.211040578581747] [321:9.454621240517747] [1218:20.024469182998658] [343:10.271546950227107] [11770:22.588724496973352] [5807:10.047320275561379] [26514:23.800264615276035] [29889:16.62579876333021] [13:14.47209195858064] [6716:19.150209322284365] [2462:12.972980729772905] [29892:11.863637377688187] [540:11.407205977456128] [471:12.026429451611941] [22049:9.843769819087358] [297:13.912187629375182] [278:13.992672100063404] [16423:14.024325453276948] [746:13.018670797002853] [540:15.09691628243275] [4446:14.45173913411657] 
    

logit 값에 0.5% 정도 오차가 있다.

tokenizer.decode

디코드 결과 (CUDA)

I have a dream. He dreams of a big, beautiful garden full of flowers and trees. He dreams of playing with his friends and eating yummy snacks.
One day, he was walking in the garden when he saw

복수형으로 처리되는 차이가 있음.

디코드 결과 (Python)

I have a dream. He dream of a big, beautiful garden full of flower and tree. He dream of playing with hi friend and eating yummy snack.
One day, he wa walking in the garden when he saw

속도

  • 4080 SUPER, cuBLAS, stories15M.bin: Token count: 50, elapsed: 0.017000s, 2823 tokens/s
  • 4080 SUPER, naive matmul, stories15M.bin: Token count: 50, elapsed: 0.034000s, 1441 tokens/s
  • 4080 SUPER, CPU(NumPy), stories15M.bin: 42 tokens/s
  • 4080 SUPER, cuBLAS, stories110M.bin: Token count: 50, elapsed: 0.050000s, 1000 tokens/s
  • 4080 SUPER, naive matmul, stories110M.bin: Token count: 50, elapsed: 0.433000s, 247 tokens/s

Last Modified: 2024/10/29 18:57:44

is a collection of Papers I have written.
© 2000 - Sang Park Except where otherwise noted, content on this site is licensed under a CC BY 4.0.
This site design was brought from Distill.