java - What is the principle ? When Spark deal with data bigger than memory capacity? -

-

as know , spark use memory cache data , compute data in memory.but if data bigger memory? read source code ,but don't know class schedule job? or explain principle of how spark deal question?

om-nom-nom gave answer, comment reason, thought i'd post actual answer:

https://spark.apache.org/docs/latest/scala-programming-guide.html#rdd-persistence


Comments

Post a Comment

Popular posts from this blog

java - Intellij Synchronizing output directories .. -

-
i trying build project, run in debug mode , keep getting "synchronizing output directories" in intellij. takes around 5-10 minutes actual build. project empty , stored on local drive. tried invalidate catche has not worked. out of ideas .. had similar issues? most likely, of data on network drive or in profile being synchronized network drive. if on windows, try moving .ideaxxx/system directory , work directory somewhere else on c: drive.
Read more

git - Initial Commit: "fatal: could not create leading directories of ..." -

-
i trying make initial commit git repository on github unity project. followed along this guide am. note: reason or another, couldn't set unity's asset serialization mode force text, settled on mixed (which believe should work). when call git clone --bare . , error. note not creator of repository , contributor (though making initial commit). here's in terminal (i'm using git bash): welcome git (version 1.8.4-preview20130916) run 'git git' display index. run 'git <command>' display specific commands. cheddar@cheddar-pc ~ $ cd documents/ics168swarch/ cheddar@cheddar-pc ~/documents/ics168swarch $ git init initialized empty git repository in c:/users/cheddar/documents/ics168swarch/.git / cheddar@cheddar-pc ~/documents/ics168swarch (master) $ git add . warning: lf replaced crlf in assets/prefabs.meta. file have original line endings in working directory. warning: lf replaced crlf in assets/prefabs/pellet.prefab.meta. file have original
Read more