 |
|
| node: | update |
| template: | 4 |
| parent: | 28.09.2022-8:27:00 |
| owner: | drakh |
| created: | 29.09.2022 - 19:38:20 |
|
cwbe coordinatez: 101 63533 63608 8771344 9005291 9005828 ABSOLUT KYBERIA |
| permissions | |
| you: | r, |
| system: | public |
| net: | yes |
total descendants::0
total children::0
tak, rozobral som si ten jeho "chargpt" ukazku a upravil som si to tak aby mi to fungovalo s BPE encodingom (teda ta ista tokenizacia ako GPT)
s block_size=1024 (dlzka kontextu co vie drzat), to iste co GPT-2 a batch_size=1, ak som nasiel spravny udaj tak OpenAI pouzilo pri GPT-2 bacth_size=512, mi to vie trenovat 124M model from scratch na mojej grafike..
s block_size=1024 (dlzka kontextu co vie drzat), to iste co GPT-2 a batch_size=1, ak som nasiel spravny udaj tak OpenAI pouzilo pri GPT-2 bacth_size=512, mi to vie trenovat 124M model from scratch na mojej grafike..