chakagadzirwa Intelligence
Supercharging Graph Neural Networks ine Mitauro Yakakura Mutauro: Iyo Yekupedzisira Gwaro
Magirafu zvimiro zvedata zvinomiririra hukama hwakaoma munzvimbo dzakasiyana siyana, kusanganisira masocial network, mabhesi eruzivo, biological system, nezvimwe zvakawanda. Mune magirafu aya, masangano anomiririrwa semanodhi, uye hukama hwavo hunoratidzwa semipendero.
Iko kugona kunyatso miririra uye kufunga nezve izvi zvakaomarara hukama zvimiro zvakakosha pakugonesa kufambira mberi muminda senge network sainzi, cheminformatics, uye vanokurudzira masisitimu.
Graph Neural Networks (GNNs) yakabuda seyakasimba yakadzika yekudzidza chimiro chegirafu muchina wekudzidza mabasa. Nekubatanidza iyo graph topology mune neural network architecture kuburikidza nevavakidzani kuunganidzwa kana magirafu convolutions, maGNN anogona kudzidza yakaderera-dimensional vector inomiririra iyo inokodha ese manode maficha uye maitiro avo ezvimiro. Izvi zvinobvumira maGNNs kuti awane kuita-kwe-the-art kuita pamabasa senge node classification, fungidziro yekubatanidza, uye kurongedza magirafu munzvimbo dzakasiyana dzekushandisa.
Nepo maGNN akafambisa kufambira mberi kwakakura, mamwe matambudziko akakosha achiripo. Kuwana data rakanyorwa zvemhando yepamusoro yekudzidzira modhi inotariswa yeGNN kunogona kudhura uye kunotora nguva. Pamusoro pezvo, maGNN anogona kunetsekana neakasiyana magirafu zvimiro uye mamiriro apo kugoverwa kwegirafu panguva yekuyedzwa kunosiyana zvakanyanya kubva kudhata rekudzidziswa (kunze-kwe-kugovera generalization).
Mukufanana, Makuru Mutauro Models (LLMs) seGPT-4, uye Anofona vatora nyika nedutu nekunzwisisa kwavo kunoshamisa kwemutauro wechisikigo uye kugona kwechizvarwa. Vakadzidziswa pane makuru mameseji corpora ane mabhiriyoni emaparamita, maLLM anoratidza anoshamisa mashoma-kupfura hunyanzvi hwekudzidza, generalization pamabasa ese, uye commonsense hunyanzvi hwekufunga hwaimbofungidzirwa kuve hwakanyanya kunetsa kune AI masisitimu.
Kubudirira kukuru kweLLM kwakakonzera ongororo mukusimudzira simba radzo rekuita mabasa ekudzidza muchina wegraph. Kune rimwe divi, ruzivo uye kugona kufunga kweLLM kunopa mikana yekusimudzira echinyakare maGNN modhi. Sezvineiwo, mamiririro akarongwa uye ruzivo rwechokwadi rwuri mumagirafu runogona kubatsira mukugadzirisa zvimwe zvipimo zvakakosha zveLLMs, sekufungidzira uye kushaya kududzira.
Muchinyorwa chino, isu tichaongorora tsvagiridzo yazvino pamharadzano yekudzidza muchina wegraph uye mhando dzemitauro mikuru. Tichaongorora kuti maLLM angashandiswa sei kusimudzira zvakasiyana-siyana zvegraph ML, nzira dzekuongorora dzekubatanidza ruzivo rwegirafu muLLM, uye kukurukura mashandisirwo ari kubuda uye mafambiro emangwana eiyi ndima inonakidza.
Graph Neural Networks uye Kuzvitarisira Kudzidza
Kuti tipe mamiriro anodiwa, isu tichatanga taongorora muchidimbu pfungwa dzepakati uye nzira mugirafu neural network uye inozvitarisira yega girafu inomiririra kudzidza.
Grafu Neural Network Architectures
Graph Neural Network Architecture - mabviro
Musiyano wakakosha pakati pechinyakare yakadzika neural network uye maGNNs uri mukukwanisa kwavo kushanda zvakananga pane graph-yakarongeka data. MaGNNs anotevera nharaunda yekuunganidza chirongwa, uko imwe neimwe node yakaunganidzwa inoratidzira mavheji kubva kune vavakidzani vayo kuti iverenge inomiririra yayo.
Mazhinji maGNN ekuvaka akakurudzirwa neakasiyana emeseji meseji uye mabasa ekugadzirisa, senge. Grafu Convolutional Networks (GCNs), GraphSAGE, Grafu Attention Networks (GATs), uye Grafu Isomorphism Networks (GINs) pakati pevamwe.
Munguva pfupi yapfuura, magirafu ekushandura akawana mukurumbira kuburikidza nekugadzirisa maitiro ekuzvitarisa kubva kumutauro wechisikigo anoshandura kuti ashande pane graph-structured data. Mimwe mienzaniso inosanganisira GraphormerTransformer, uye GraphFormers. Aya mamodheru anokwanisa kubata-refu-refu kutsamira pagirafu zvirinani pane zvakangoita muvakidzani-based GNNs.
Kuzvitarisira Kudzidza paMagirafu
Nepo maGNN ari mamodheru ane simba ekumiririra, kuita kwavo kunowanzo kuvharirwa nekushaikwa kwemaseti makuru akanyorwa anodiwa pakudzidziswa kunotariswa. Kudzidza kwekuzvitarisira kwabuda separadigm inovimbisa kufanodzidzisa maGNN pane isina kunyorwa data regirafu nekusimudzira mabasa ekufungidzira anongoda iyo mukati megirafu chimiro uye node maficha.
Mamwe mabasa akajairwa ekunyepedzera anoshandiswa kuzvitarisira wega GNN pre-training anosanganisira:
- Node Property Prediction: Kungovharisa kana kushatisa chikamu cheunhu/zvimiro uye kupa GNN basa rekuzvivaka patsva.
- Edge/Link Prediction: Kudzidza kufanotaura kana mupendero uripo pakati pemanodhi, kazhinji zvichienderana nekusarudzika kumucheto masking.
- Contrastive Learning: Kukwirisa kufanana pakati pemagirafu maonero eiyo yakafanana girafu sampuli uchisundira kunze maonero kubva kune akasiyana magirafu.
- Mutual Information Maximization: Kukwirisa ruzivo rwekuwirirana pakati pezvinomiririra node dzemuno uye chinomiririra chinotarisirwa senge girafu repasi rose.
Mabasa ekunyepedzera seaya anobvumira iyo GNN kuti itore zvine musoro zvimiro uye semantic mapatani kubva kune isina kunyorwa data yegraph panguva yepre-kudzidziswa. Iyo GNN isati yadzidziswa inogona kugadziridzwa zvakanaka pamadiki madiki akanyorwa kuti ibudirire pamabasa akasiyana-siyana ekudzika senge node classification, fungidziro yekubatanidza, uye girafu kupatsanura.
Nekusimudzira kuzvitarisira, maGNN akafanodzidziswa pamaseti makuru asina kunyorwa anoratidza zvirinani kujekeswa, kusimba kumashifiti ekugovera, uye kugona zvichienzaniswa nekudzidziswa kubva pakutanga. Nekudaro, zvimwe zvipimo zvakakosha zvechinyakare GNN-yakavakirwa-yega nzira dzekuzvitarisira dzakasara, izvo zvatichaongorora zvinogonesa maLLM kugadzirisa zvinotevera.
Kuvandudza Girafu ML neMienzaniso Yakakura Mutauro
Kubatanidzwa kweGrafu uye LLM - mabviro
Kugona kunoshamisa kweLLM mukunzwisisa mutauro wechisikigo, kufunga, uye kushoma-pfuti kudzidza kunopa mikana yekusimudzira akawanda mapeipi ekudzidza muchina wegraph. Isu tinoongorora mamwe akakosha ekutsvaga nzira munzvimbo ino:
Dambudziko rakakosha pakushandisa maGNN kuwana emhando yepamusoro inomiririra yemanodhi nemicheto, kunyanya kana iine hupfumi hwemavara sematsananguro, mazita, kana abstracts. Nechinyakare, nyore bhegi-re-mazwi kana kufanodzidziswa mazwi ekumisikidza mamodheru akashandiswa, ayo anowanzo kutadza kutora iyo nuanced semantics.
Mabasa achangoburwa aratidza simba rekushandisa mamodheru emitauro mikuru semakodhi emavara ekugadzira zviri nani node/mupendero maficha anomiririra asati apfuudza kuGNN. Semuyenzaniso, Chen et al. shandisa maLLM seGPT-3 kuvharidzira mameseji node hunhu, kuratidza kwakakosha kuita kwakawanda pamusoro pechinyakare kuisirwa mazwi pane node classification mabasa.
Kupfuura zvirinani maencoder emavara, maLLM anogona kushandiswa kugadzira yakawedzera ruzivo kubva kune ekutanga mameseji maitiro nenzira inotariswa semi-inotariswa. Tepi inogadzira zvinyorwa zvingangoitika / tsananguro dzemanodhi uchishandisa LLM uye inoshandisa izvi semamwe akawedzera maficha. KEA inobvisa mazwi kubva kumavara hunhu uchishandisa LLM uye inowana yakadzama tsananguro yeaya mazwi kuti awedzere maficha.
Nekuvandudza kunaka uye kujekeswa kwezvimiro zvekupinza, LLMs dzinogona kupa kugona kwavo kwepamusoro kwekunzwisisa mutauro wechisikigo kuGNNs, zvichisimudzira mashandiro pamabasa epasi.
Kuderedza Kutsamira paData Yakanyorwa
Mukana wakakosha weLLMs kugona kwavo kuita zvine musoro pamabasa matsva vasina data rakanyorwa, nekuda kwekudzidziswa kwavo kwekutanga pane yakakura zvinyorwa corpora. Izvi zvishoma-zvipfuti zvekudzidza zvinogona kukwidziridzwa kudzikamisa kuvimba kweGNNs pane makuru akanyorwa dataset.
Imwe nzira ndeyekushandisa maLLM kuita fungidziro yakananga pamabasa egirafu nekutsanangura chimiro chegirafu uye ruzivo rwenode mumutauro wechisikigo. Nzira dzakadai RairiraGLM uye GPT4Graph nyatsogadzirisa maLLM akaita seLLaMA neGPT-4 uchishandisa zvirevo zvakanyatsogadzirwa zvinosanganisa ruzivo rwemagraph topology sekubatanidza node, nharaunda nezvimwe. MaLLM akarongwa anogona kubva agadzira fungidziro yemabasa akaita sekuronga kwenodhi uye kufanotaura kwekubatanidza nenzira ye zero-pfuti panguva yekufungidzira.
Ndichiri kushandisa maLLM seatema-bhokisi akafanotaura kwakaratidza kuvimbisa, kuita kwavo kunodzikisira kune mamwe akaomarara magirafu mabasa uko kuenzanisira kwakajeka kwechimiro kunobatsira. Dzimwe nzira dzinoshandisa maLLM pamwe chete nemaGNN - iyo GNN inokodha chimiro chegirafu nepo LLM ichipa kunzwisiswa kwesemantic kwemanodhi kubva kutsananguro dzezvinyorwa.
Grafu Kunzwisisa neLLM Framework - mabviro
GraphLLM inoongorora nzira mbiri: 1) LLMs-as-Enhancers apo maLLMs anokodha zvinyorwa zvinyorwa zvinyorwa zvisati zvapfuura kuGNN, uye 2) LLMs-as-Predictors apo LLM inotora zvinomiririra zvepakati zveGNN sechipo chekuita kufanotaura kwekupedzisira.
GLEM inoenderera mberi nekukurudzira musiyano weEM algorithm iyo inochinjana pakati pekuvandudza iyo LLM neGNN zvikamu zvekusimudzirana.
Nekudzikisa kuvimba nedata rakanyorwa kuburikidza neashoma-kupfura hunyanzvi uye semi-inotariswa kuwedzera, LLM-yakakwidziridzwa girafu nzira dzekudzidza dzinogona kuvhura maapplication matsva nekuvandudza kugona kwedata.
Kuvandudza maLLM nemaGrafu
Nepo maLLM akabudirira zvakanyanya, achiri kutambura kubva kune zvakakosha zvisingakwanisi sekufungidzira (kuburitsa zvisiri zvechokwadi), kushaikwa kwekududzira mumaitiro avo ekufunga, uye kusakwanisa kuchengetedza ruzivo rwechokwadi rwakafanana.
Magirafu, kunyanya magirafu eruzivo anomiririra ruzivo rwechokwadi rwakarongeka kubva kune akavimbika masosi, anopa nzira dzinovimbisa dzekugadzirisa zvikanganiso izvi. Isu tinoongorora dzimwe nzira dzinobuda munzira iyi:
Ruzivo Girafu Yakanatsiridza LLM Pre-kudzidziswa
Zvakafanana nemabatiro anoita maLLM akafanodzidziswa pane yakakura zvinyorwa corpora, mabasa achangoburwa vakaongorora vasati vavadzidzisa pamagirafu ezivo kuti vawedzere ruzivo rwechokwadi uye kugona kufunga.
Dzimwe nzira dzinoshandura data rekuisa nekungobatanidza kana kubatanidza chokwadi cheKG chinopetwa katatu nemavara emutauro chaiwo panguva yekudzidzira kusati kwaitwa. E-BERT inogadzirisa KG entity vectors neBERT's wordpiece embeddings, ukuwo K-BERT ichivaka miti ine mutsara wekutanga uye yakakodzera KG katatu.
Basa reLLMs muGrafu Machine Kudzidza:
Vatsvagiri vakaongorora nzira dzakati wandei dzekubatanidza maLLM mupombi yekudzidza magirafu, imwe neimwe iine zvayakanakira uye mashandisiro ayo. Heano mamwe emabasa ane mukurumbira anogona kutamba neLLM:
- LLM seEnhancer: Mukuita uku, maLLM anoshandiswa kupfumisa zvinyorwa zvemavara zvine chekuita nemanodhi muTAG. Kugona kweLLM kuburitsa tsananguro, masangano eruzivo, kana pseudo-labels zvinogona kuwedzera ruzivo rwesemantic rwunowanikwa kuGNN, zvichitungamira kune yakagadziridzwa inomiririra inomiririra uye yakadzika basa rekuita.
Semuyenzaniso, iyo TAPE (Text Augmented Pre-trained Encoders) modhi inosimudzira ChatGPT kugadzira tsananguro nemanyepo emapepa etiweki etiweki, ayo anobva ashandiswa kugadzirisa modhi yemutauro. Izvo zvinokonzeresa kuisirwa zvinopihwa muGNN yekuisa node kupatsanura uye kubatanidza mabasa ekufanotaura, kuwana mamiriro-e-the-art mhedzisiro.
- LLM sePredictor: Panzvimbo pekusimudzira maficha ekuisa, dzimwe nzira dzinoshandisa zvakananga maLLM sechinhu chekufungidzira chemabasa ane chekuita negraph. Izvi zvinosanganisira kushandura chimiro chegirafu kuita chinyorwa chinomiririra chinogona kugadziriswa neLLM, icho chinobva chaburitsa chinodiwa chinobuda, senge node label kana graph-level fungidziro.
Mumwe muenzaniso unozivikanwa ndeyeGPT4Graph modhi, inomiririra magirafu achishandisa Graph Modelling Mutauro (GML) uye inosimudzira ine simba GPT-4 LLM yezero-kupfura girafu mabasa ekufunga.
- GNN-LLM Kurongeka: Imwe mutsara wekutsvagisa wakanangana nekuenzanisa nzvimbo dzekumisikidza dzeGNNs neLLMs, zvichibvumira kubatanidzwa kusina musono kweruzivo rwezvimiro uye semantic. Nzira idzi dzinobata GNN neLLM senzira dzakaparadzana uye dzinoshandisa matekiniki akaita sekudzidza kwakasiyana kana kuti distillation kuti vaenzanise zvinomiririra.
The MoleculeSTM modhi, semuenzaniso, inoshandisa chinangwa chakasiyana kuenzanisa zvakamisikidzwa muGNN neLLM, zvichiita kuti LLM ibatanidze ruzivo rwezvimiro kubva kuGNN ukuwo GNN ichibatsirika kubva muruzivo rweLLM.
Matambudziko Nekugadzirisa
Nepo kubatanidzwa kweLLM uye kudzidza magirafu kune chivimbiso chikuru, matambudziko akati wandei anofanirwa kugadziriswa:
- Kubudirira uye Scalability: MaLLM ane mukurumbira wekushandisa-yakawanda, kazhinji inoda mabhiriyoni emaparamita uye rakakura simba remakomputa rekudzidzisa uye kufungidzira. Izvi zvinogona kuve bhodhoro rakakosha rekuisa LLM-yakakwidziridzwa magirafu modhi mune chaiyo-yepasirese maapplication, kunyanya pazvishandiso zvinomanikidzirwa.
Imwe inovimbisa mhinduro ndeye ruzivo distillation, uko ruzivo kubva kuLLM yakakura (mudzidzisi muenzaniso) inotamirwa kune diki, inobudirira GNN (mudzidzi muenzaniso).
- Data Leakage uye Kuongorora: LLMs dzakafanodzidziswa pahuwandu hwakawanda hwe data inowanikwa pachena, iyo inogona kusanganisira bvunzo seti kubva kune akajairwa benchmark datasets, zvichitungamira kune inogona kudonha data uye yakawandisa mashandiro. Vatsvaguri vatanga kuunganidza dhatabheti nyowani kana sampling bvunzo data kubva panguva mushure mekudzidziswa kweLLM kudzikamisa nyaya iyi.
Pamusoro pezvo, kumisikidza mabhenji ekuongorora akaenzana uye akazara emhando dzekudzidza magirafu akakwidziridzwa eLLM kwakakosha kuyera kugona kwavo kwechokwadi uye kugonesa kuenzanisa kune musoro.
- Transferability uye Explainability: Nepo maLLM achikunda pa zero-kupfura uye mashoma-kupfura kudzidza, kugona kwavo kuendesa ruzivo kune akasiyana siyana magirafu madomasi uye zvimiro zvinoramba zviri dambudziko rakavhurika. Kuvandudza kuchinjika kweaya mamodheru idanho rakakosha rekutsvagisa.
Uyezve, kusimudzira kutsanangurwa kweLLM-yakavakirwa magirafu emhando yekudzidza kwakakosha pakuvaka kuvimba uye kugonesa kutorwa kwavo muzvidzidzo zvepamusoro. Kushandisa hunyanzvi hwekufunga hweLLMs kuburikidza nehunyanzvi hwakadai chain-of-pfungwa kukurudzira inogona kubatsira pakuvandudza kutsanangura.
- Multimodal Integration: Magirafu anowanzo kuve nezvakawanda kwete zvemashoko chete, ane node uye mipendero ingangove yakabatana neakasiyana modalities, senge mifananidzo, odhiyo, kana manhamba data. Kuwedzera kubatanidzwa kweLLM kune idzi multimodal graph marongero kunopa mukana unonakidza wekutsvaga mune ramangwana.
Chaiyo-yenyika Zvikumbiro uye Nyaya Dzidzo
Iko kubatanidzwa kweLLMs uye girafu muchina kudzidza kwatoratidza mhedzisiro inovimbisa mune dzakasiyana-siyana dzepasirese application:
- Molecular Property Prediction: Mundima yecomputational chemistry uye kuwanikwa kwezvinodhaka, maLLM akashandiswa kusimudzira kufanotaura kwemamorekuru nekubatanidza ruzivo rwezvimiro kubva kumagirafu emokuru. The Muenzaniso weLLM4Mol, semuenzaniso, inosimudzira ChatGPT kugadzira tsananguro dzeSMILES (Simplified Molecular-Input Line-Entry System) inomiririra mamorekuru, ayo anobva ashandiswa kuvandudza huchokwadi hwemabasa ekufanotaura pfuma.
- Ruzivo Girafu Kupedzwa uye Kukurukurirana: Ruzivo rwemagirafu imhando yakakosha yechimiro chegirafu chinomiririra masangano epasirese uye hukama hwavo. MaLLM akaongororwa mabasa senge ruzivo rwegirafu kupedzisa uye kufunga, apo chimiro chegirafu uye ruzivo rwezvinyorwa (semuenzaniso, tsananguro yesangano) inoda kutariswa pamwe chete.
- Anokurudzira Maitiro: Munzvimbo yeanokurudzira masisitimu, zvimiro zvegirafu zvinowanzo shandiswa kumiririra mushandisi-chinhu chekudyidzana, nemanodhi anomiririra vashandisi uye zvinhu, uye mipendero inoratidza kudyidzana kana kufanana. MaLLM anogona kukwidziridzwa kuti awedzere magirafu aya nekugadzira mushandisi/chinhu chedivi ruzivo kana kusimbisa mipendero yekudyidzana.
mhedziso
Kubatana pakati peMakuru Mutauro Models uye Graph Machine Kudzidza inopa muganho unonakidza mukutsvagisa hungwaru hwekugadzira. Nekusanganisa kurerekera kwemaitiro eGNN ane simba rekunzwisisa kweLLMs, tinokwanisa kuvhura mikana mitsva mumabasa ekudzidza magirafu, kunyanya pamagirafu akanyorwa.
Nepo kufambira mberi kwakakosha kwaitwa, matambudziko anoramba ari munzvimbo dzakaita sekuita, scalability, kutamisa, uye kutsanangura. Tekiniki dzakaita senge zivo distillation, mabhenji ekuongorora zvakanaka, uye kubatanidzwa kwemultimodal zviri kugadzira nzira yekuendesa kunoshanda kweLLM-akakwenenzverwa magirafu emhando dzekudzidza mumashandisirwo epasirese.
