/home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpb0Fh5u /home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpb0Fh5u *************** RDDL-PARSER CALL WITH 600SEC *************** Parsing... Setting outcome pruning to 0.5 ...finished (0.00101304s). instantiating... Instantiating variables... ...finished (9.05991e-06) Instantiating CPFs... ...finished (3.91006e-05) Instantiating preconditions... ...finished (5.00679e-06) ...finished (6.38962e-05s). preprocessing... Preparing evaluatables... ...finished (3.38554e-05) Preparing actions... ...finished (3.40939e-05) Calculating CPF domain... ...finished (2.21729e-05) Finalizing evaluatables... ...finished (2.09808e-05) Computing determinization... ...finished (2.69413e-05) Determining task properties... ...finished (1.90735e-06) Preparing hash keys... ...finished (7.86781e-06) Precomputing evaluatables... ...finished (0.000161886) Calculating min and max reward... ...finished (1.90735e-06) ...finished (0.000348806s). analyzing task... Creating training set with 90 candidates. ...finished (0.00167608s). writing output for instance dice_game_demo_inst_mdp__1... ...finished (0.00127792s). writing transition relations to json file... ...finished (0.000281811s). total time: 0.00469494s RDDL-Parser took: 0.0122084s learning... THTS: learning... DD_Heuristic: learning [31s (0.125%)] with /home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpb0Fh5u/dice_game_demo_inst_mdp__1.json... Horizon: 10 Round to dezimal: 2 { "actions": { "noop": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==s2])))" }, "roll(d1) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==0] * (1) + [s0_primed==1] * (1) + [s0_primed==2] * (1) + [s0_primed==3] * (1) + [s0_primed==4] * (1) + [s0_primed==5] * (1))) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==s2])))" }, "roll(d2) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==0] * (1) + [s1_primed==1] * (1) + [s1_primed==2] * (1) + [s1_primed==3] * (1) + [s1_primed==4] * (1) + [s1_primed==5] * (1))) * (1 * ([s2_primed==s2])))" }, "roll(d3) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==0] * (1) + [s2_primed==1] * (1) + [s2_primed==2] * (1) + [s2_primed==3] * (1) + [s2_primed==4] * (1) + [s2_primed==5] * (1))))" } }, "goal_state": { "fake_goal": 1 }, "initial_state": { "fake_goal": 0, "s0": 0, "s1": 0, "s2": 0 }, "variables": { "fake_goal": { "domain": 2 }, "s0": { "domain": 6 }, "s1": { "domain": 6 }, "s2": { "domain": 6 } } } Original ordering: s0 s1 s2 fake_goal Build ast.....done! Compute fan-in...done! Fan-in ordering: fake_goal s0 s1 s2 [s0 : 1] [s1 : 2] [s2 : 3] [fake_goal : 0] Num variables: 4 => 10 [ incl. primed: 20 ] noop......overall time: 0.04 => Time left: 31.21s roll(d1) ......overall time: 0.06 => Time left: 31.19s roll(d2) ......overall time: 0.07 => Time left: 31.18s roll(d3) ......overall time: 0.08 => Time left: 31.17s Plan step 1/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.08 => Time left: 31.17s Plan step 2/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 3/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 4/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 5/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 6/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 7/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 8/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 9/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 10/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Completed layers: 11 Reset Det Task. ... finished THTS: ...finished ...finished (0.246356s). Final task: ----------------Actions--------------- Action fluents: roll(d1) roll(d2) roll(d3) --------------- Legal Action Combinations: noop() : Index : 0 Relevant preconditions: --------------- roll(d3) : Index : 1 Relevant preconditions: --------------- roll(d2) : Index : 2 Relevant preconditions: --------------- roll(d1) : Index : 3 Relevant preconditions: --------------- -----------------CPFs----------------- die-value(d1) HashIndex: 0, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d1) : 1 Formula: case roll(d1) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d1) Determinized formula: case roll(d1) then 0 case 1 then die-value(d1) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5 KleeneHashKeyBase: 1 -------------- die-value(d2) HashIndex: 1, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d2) : 1 Formula: case roll(d2) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d2) Determinized formula: case roll(d2) then 0 case 1 then die-value(d2) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 6, 2: 12, 3: 18, 4: 24, 5: 30 KleeneHashKeyBase: 63 -------------- die-value(d3) HashIndex: 2, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d3) : 1 Formula: case roll(d3) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d3) Determinized formula: case roll(d3) then 0 case 1 then die-value(d3) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 36, 2: 72, 3: 108, 4: 144, 5: 180 KleeneHashKeyBase: 3969 -------------- Reward CPF: Reward HashIndex: 3, deterministic, caching in vectors, Kleene caching in maps. Action Hash Key Map: Formula: (+ (== die-value(d1) 0) (* (== die-value(d1) 1) 2) (* (== die-value(d1) 2) 3) (* (== die-value(d1) 3) 4) (* (== die-value(d1) 4) 5) (* (== die-value(d1) 5) 6) (== die-value(d2) 0) (* (== die-value(d2) 1) 2) (* (== die-value(d2) 2) 3) (* (== die-value(d2) 3) 4) (* (== die-value(d2) 4) 5) (* (== die-value(d2) 5) 6) (== die-value(d3) 0) (* (== die-value(d3) 1) 2) (* (== die-value(d3) 2) 3) (* (== die-value(d3) 3) 4) (* (== die-value(d3) 4) 5) (* (== die-value(d3) 5) 6) ) Minimal reward: 3 Maximal reward: 18 Is action independent: 1 ------State Fluent Hash Key Map------- a change of probabilistic state fluent 0 influences variables 0 (2) 3 (1) a change of probabilistic state fluent 1 influences variables 1 (2) 3 (6) a change of probabilistic state fluent 2 influences variables 2 (2) 3 (36) a change of variable 0 influences variables in Kleene states 0 (2) 3 (1) a change of variable 1 influences variables in Kleene states 1 (2) 3 (63) a change of variable 2 influences variables in Kleene states 2 (2) 3 (3969) ---------Action Preconditions--------- ----------Initial State--------------- die-value(d1): 0 die-value(d2): 0 die-value(d3): 0 Remaining Steps: 10 StateHashKey: 0 Hashing of States is possible. Hashing of KleeneStates is possible. No reward locks detected in the training phase. This task contains unreasonable actions only in the determinization. The final reward is determined by applying NOOP. *********************************************** >>> STARTING ROUND 1 -- REMAINING TIME 249s *********************************************** *********************************************** Planning step 1/10 in round 1/10 Current state: | 0 0 0 Setting time for this decision to 2.46687s. THTS: Maximal search depth set to 10 Search time: 2.46688s Statistics of THTS: Performed trials: 595786 Created SearchNodes: 2978935 Cache Hits: 0 Action Selection: Exploitation in Root: 228336 Exploration in Root: 367450 Percentage Exploration in Root: 0.616748 Skipped backups: 3887274 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 117.26 (in 595790 real visits) Q-Value Estimates: noop() : 109.303 (in 3995 real visits) roll(d3) : 117.241 (in 193468 real visits) roll(d2) : 117.256 (in 199747 real visits) roll(d1) : 117.26 (in 198580 real visits) Used RAM: 502640 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 1/10 Current state: | 0 0 0 Setting time for this decision to 2.46686s. THTS: Maximal search depth set to 9 Search time: 2.46686s Statistics of THTS: Performed trials: 553675 Created SearchNodes: 2768160 Cache Hits: 0 Skipped backups: 7456836 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 99.2602 (in 553679 real visits) Q-Value Estimates: noop() : 91.6019 (in 3715 real visits) roll(d3) : 99.2458 (in 182058 real visits) roll(d2) : 99.2452 (in 182158 real visits) roll(d1) : 99.2602 (in 185748 real visits) Used RAM: 502640 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 1/10 Current state: | 2 0 0 Setting time for this decision to 2.46686s. THTS: Maximal search depth set to 8 Search time: 2.46686s Statistics of THTS: Performed trials: 551608 Created SearchNodes: 2668055 Cache Hits: 7661 Skipped backups: 11149140 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 88.4421 (in 551612 real visits) Q-Value Estimates: noop() : 81.7606 (in 5387 real visits) roll(d3) : 88.4421 (in 269334 real visits) roll(d2) : 88.4286 (in 261032 real visits) roll(d1) : 86.5011 (in 15859 real visits) Used RAM: 502640 Submitted action: roll(d3) Immediate reward: 5 *********************************************** *********************************************** Planning step 4/10 in round 1/10 Current state: | 2 0 5 Setting time for this decision to 2.46685s. THTS: Maximal search depth set to 7 Search time: 0.795405s Statistics of THTS: Performed trials: 233533 Created SearchNodes: 533664 Cache Hits: 105926 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 96.6667 (in 233537 real visits) Q-Value Estimates: noop() : SOLVED with: 90.8333 (in 4208 real visits) roll(d3) : SOLVED with: 78.3038 (in 4297 real visits) roll(d2) : SOLVED with: 96.6667 (in 210319 real visits) roll(d1) : SOLVED with: 93.9687 (in 14713 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 1/10 Current state: | 2 1 5 Setting time for this decision to 2.48425s. THTS: Maximal search depth set to 6 Search time: 0.0355895s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 81.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 77.375 (in 2 real visits) roll(d3) : SOLVED with: 66.6624 (in 7 real visits) roll(d2) : SOLVED with: 81.8333 (in 7 real visits) roll(d1) : SOLVED with: 80.3395 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 1/10 Current state: | 2 1 5 Setting time for this decision to 2.51002s. THTS: Maximal search depth set to 5 Search time: 0.0181165s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 62.375 (in 2 real visits) roll(d3) : SOLVED with: 53.456 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 64.8935 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 11 *********************************************** *********************************************** Planning step 7/10 in round 1/10 Current state: | 2 5 5 Setting time for this decision to 2.53652s. THTS: Maximal search depth set to 4 Search time: 0.0181883s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 63.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 61.75 (in 2 real visits) roll(d3) : SOLVED with: 55.375 (in 7 real visits) roll(d2) : SOLVED with: 55.375 (in 7 real visits) roll(d1) : SOLVED with: 63.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 1/10 Current state: | 3 5 5 Setting time for this decision to 2.5636s. THTS: Maximal search depth set to 3 Search time: 0.018122s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 48 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 48 (in 2 real visits) roll(d3) : SOLVED with: 43.75 (in 7 real visits) roll(d2) : SOLVED with: 43.75 (in 7 real visits) roll(d1) : SOLVED with: 47.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 9/10 in round 1/10 Current state: | 3 5 5 Setting time for this decision to 2.59126s. THTS: Maximal search depth set to 2 Search time: 0.0181155s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 12710838 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 29.5 (in 7 real visits) roll(d2) : SOLVED with: 29.5 (in 7 real visits) roll(d1) : SOLVED with: 31.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 1/10 Current state: | 3 5 5 Setting time for this decision to 2.61953s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 7 Accumulated number of trials in root state: 595786 Accumulated number of search nodes in root state: 2978935 Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 1 -- REWARD RECEIVED: 106 *********************************************** *********************************************** >>> STARTING ROUND 2 -- REMAINING TIME 241s *********************************************** *********************************************** Planning step 1/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.64843s. THTS: Maximal search depth set to 10 Search time: 2.64844s Statistics of THTS: Performed trials: 743588 Created SearchNodes: 1929105 Cache Hits: 428989 Action Selection: Exploitation in Root: 261152 Exploration in Root: 482436 Percentage Exploration in Root: 0.648795 Skipped backups: 17407874 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 112.542 (in 743592 real visits) Q-Value Estimates: noop() : 107.569 (in 5185 real visits) roll(d3) : 112.514 (in 242243 real visits) roll(d2) : 112.542 (in 259232 real visits) roll(d1) : 112.505 (in 236932 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.64842s. THTS: Maximal search depth set to 9 Search time: 0.0989982s Statistics of THTS: Performed trials: 9512 Created SearchNodes: 14391 Cache Hits: 8079 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 9516 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 335 real visits) roll(d3) : SOLVED with: 95.0692 (in 3109 real visits) roll(d2) : SOLVED with: 95.0692 (in 3091 real visits) roll(d1) : SOLVED with: 95.0692 (in 2981 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.67739s. THTS: Maximal search depth set to 8 Search time: 0.0186593s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 80.2784 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68.9334 (in 2 real visits) roll(d3) : SOLVED with: 80.2784 (in 7 real visits) roll(d2) : SOLVED with: 80.2784 (in 7 real visits) roll(d1) : SOLVED with: 80.2784 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 4/10 in round 2/10 Current state: | 0 0 4 Setting time for this decision to 2.70794s. THTS: Maximal search depth set to 7 Search time: 0.0190462s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 84.9687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 77.1397 (in 2 real visits) roll(d3) : SOLVED with: 69.9334 (in 7 real visits) roll(d2) : SOLVED with: 84.9687 (in 7 real visits) roll(d1) : SOLVED with: 84.9687 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 7 *********************************************** *********************************************** Planning step 5/10 in round 2/10 Current state: | 1 0 4 Setting time for this decision to 2.7392s. THTS: Maximal search depth set to 6 Search time: 0.0185407s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 72.3395 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.8935 (in 2 real visits) roll(d3) : SOLVED with: 59.5367 (in 7 real visits) roll(d2) : SOLVED with: 72.3395 (in 7 real visits) roll(d1) : SOLVED with: 71.1397 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 8 *********************************************** *********************************************** Planning step 6/10 in round 2/10 Current state: | 1 5 4 Setting time for this decision to 2.7712s. THTS: Maximal search depth set to 5 Search time: 0.017955s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 74.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 71.375 (in 2 real visits) roll(d3) : SOLVED with: 66.8935 (in 7 real visits) roll(d2) : SOLVED with: 62.8935 (in 7 real visits) roll(d1) : SOLVED with: 74.25 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.80396s. THTS: Maximal search depth set to 4 Search time: 0.018264s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 68 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68 (in 2 real visits) roll(d3) : SOLVED with: 65.375 (in 7 real visits) roll(d2) : SOLVED with: 62.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 8/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.83753s. THTS: Maximal search depth set to 3 Search time: 0.0183157s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 51 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51 (in 2 real visits) roll(d3) : SOLVED with: 48.75 (in 7 real visits) roll(d2) : SOLVED with: 46.75 (in 7 real visits) roll(d1) : SOLVED with: 46.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 9/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.8719s. THTS: Maximal search depth set to 2 Search time: 0.0184072s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17438884 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 34 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 34 (in 2 real visits) roll(d3) : SOLVED with: 32.5 (in 7 real visits) roll(d2) : SOLVED with: 31.5 (in 7 real visits) roll(d1) : SOLVED with: 31.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 10/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.90712s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 16 Accumulated number of trials in root state: 1339374 Accumulated number of search nodes in root state: 4908040 Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** >>> END OF ROUND 2 -- REWARD RECEIVED: 105 *********************************************** *********************************************** >>> STARTING ROUND 3 -- REMAINING TIME 238s *********************************************** *********************************************** Planning step 1/10 in round 3/10 Current state: | 0 0 0 Setting time for this decision to 2.94322s. THTS: Maximal search depth set to 10 Search time: 0.0358174s Statistics of THTS: Performed trials: 9637 Created SearchNodes: 11992 Cache Hits: 8951 Action Selection: Exploitation in Root: 3945 Exploration in Root: 5692 Percentage Exploration in Root: 0.59064 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 9641 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 3217 real visits) roll(d2) : SOLVED with: 110.224 (in 3208 real visits) roll(d1) : SOLVED with: 110.224 (in 3214 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 3/10 Current state: | 0 3 0 Setting time for this decision to 2.98001s. THTS: Maximal search depth set to 9 Search time: 0.0190537s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 107.43 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.597 (in 2 real visits) roll(d3) : SOLVED with: 107.43 (in 7 real visits) roll(d2) : SOLVED with: 98.0692 (in 7 real visits) roll(d1) : SOLVED with: 107.43 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 6 *********************************************** *********************************************** Planning step 3/10 in round 3/10 Current state: | 1 3 0 Setting time for this decision to 3.01797s. THTS: Maximal search depth set to 8 Search time: 0.0182179s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 94.797 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.3632 (in 2 real visits) roll(d3) : SOLVED with: 94.797 (in 7 real visits) roll(d2) : SOLVED with: 86.6783 (in 7 real visits) roll(d1) : SOLVED with: 93.597 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 3/10 Current state: | 1 3 5 Setting time for this decision to 3.05692s. THTS: Maximal search depth set to 7 Search time: 0.018191s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 96.8333 (in 2 real visits) roll(d3) : SOLVED with: 85.3632 (in 7 real visits) roll(d2) : SOLVED with: 97.1687 (in 7 real visits) roll(d1) : SOLVED with: 100.667 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 5/10 in round 3/10 Current state: | 2 3 5 Setting time for this decision to 3.09689s. THTS: Maximal search depth set to 6 Search time: 0.0181551s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.375 (in 2 real visits) roll(d3) : SOLVED with: 73.875 (in 7 real visits) roll(d2) : SOLVED with: 83.8333 (in 7 real visits) roll(d1) : SOLVED with: 85.8333 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 6/10 in round 3/10 Current state: | 3 3 5 Setting time for this decision to 3.13795s. THTS: Maximal search depth set to 5 Search time: 0.0182417s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 71.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 70.375 (in 2 real visits) roll(d3) : SOLVED with: 63.375 (in 7 real visits) roll(d2) : SOLVED with: 71.375 (in 7 real visits) roll(d1) : SOLVED with: 71.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 14 *********************************************** *********************************************** Planning step 7/10 in round 3/10 Current state: | 3 1 5 Setting time for this decision to 3.18009s. THTS: Maximal search depth set to 4 Search time: 0.0180829s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 54.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51.75 (in 2 real visits) roll(d3) : SOLVED with: 44.9306 (in 7 real visits) roll(d2) : SOLVED with: 54.375 (in 7 real visits) roll(d1) : SOLVED with: 50.9306 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 12 *********************************************** *********************************************** Planning step 8/10 in round 3/10 Current state: | 3 4 5 Setting time for this decision to 3.2234s. THTS: Maximal search depth set to 3 Search time: 0.0180184s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 45 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45 (in 2 real visits) roll(d3) : SOLVED with: 40.75 (in 7 real visits) roll(d2) : SOLVED with: 42.75 (in 7 real visits) roll(d1) : SOLVED with: 44.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 3/10 Current state: | 3 4 5 Setting time for this decision to 3.26792s. THTS: Maximal search depth set to 2 Search time: 0.0181468s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 30 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 30 (in 2 real visits) roll(d3) : SOLVED with: 27.5 (in 7 real visits) roll(d2) : SOLVED with: 28.5 (in 7 real visits) roll(d1) : SOLVED with: 29.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 10/10 in round 3/10 Current state: | 3 4 5 Setting time for this decision to 3.31368s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 26 Accumulated number of trials in root state: 1349011 Accumulated number of search nodes in root state: 4920032 Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** >>> END OF ROUND 3 -- REWARD RECEIVED: 112 *********************************************** *********************************************** >>> STARTING ROUND 4 -- REMAINING TIME 238s *********************************************** *********************************************** Planning step 1/10 in round 4/10 Current state: | 0 0 0 Setting time for this decision to 3.36074s. THTS: Maximal search depth set to 10 Search time: 0.0180492s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 11 Exploration in Root: 8 Percentage Exploration in Root: 0.421053 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 4/10 Current state: | 4 0 0 Setting time for this decision to 3.40919s. THTS: Maximal search depth set to 9 Search time: 0.0189295s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 115.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 107.097 (in 2 real visits) roll(d3) : SOLVED with: 115.455 (in 7 real visits) roll(d2) : SOLVED with: 115.455 (in 7 real visits) roll(d1) : SOLVED with: 99.0692 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 7 *********************************************** *********************************************** Planning step 3/10 in round 4/10 Current state: | 4 0 0 Setting time for this decision to 3.45903s. THTS: Maximal search depth set to 8 Search time: 0.0180309s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 91.9687 (in 2 real visits) roll(d3) : SOLVED with: 100.097 (in 7 real visits) roll(d2) : SOLVED with: 100.097 (in 7 real visits) roll(d1) : SOLVED with: 84.2784 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 4/10 Current state: | 4 1 0 Setting time for this decision to 3.51039s. THTS: Maximal search depth set to 7 Search time: 0.0180645s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 87.1687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 80.3395 (in 2 real visits) roll(d3) : SOLVED with: 87.1687 (in 7 real visits) roll(d2) : SOLVED with: 85.9688 (in 7 real visits) roll(d1) : SOLVED with: 73.3326 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 4/10 Current state: | 4 1 1 Setting time for this decision to 3.56329s. THTS: Maximal search depth set to 6 Search time: 0.0179936s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 73.3395 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 67.8935 (in 2 real visits) roll(d3) : SOLVED with: 73.3395 (in 7 real visits) roll(d2) : SOLVED with: 73.3395 (in 7 real visits) roll(d1) : SOLVED with: 61.7365 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 9 *********************************************** *********************************************** Planning step 6/10 in round 4/10 Current state: | 4 1 4 Setting time for this decision to 3.61783s. THTS: Maximal search depth set to 5 Search time: 0.0182318s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 69.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 66.375 (in 2 real visits) roll(d3) : SOLVED with: 61.8935 (in 7 real visits) roll(d2) : SOLVED with: 69.25 (in 7 real visits) roll(d1) : SOLVED with: 61.8935 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 12 *********************************************** *********************************************** Planning step 7/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.67406s. THTS: Maximal search depth set to 4 Search time: 0.0178887s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 60 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 60 (in 2 real visits) roll(d3) : SOLVED with: 57.375 (in 7 real visits) roll(d2) : SOLVED with: 57.375 (in 7 real visits) roll(d1) : SOLVED with: 57.375 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.73208s. THTS: Maximal search depth set to 3 Search time: 0.0179693s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 45 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45 (in 2 real visits) roll(d3) : SOLVED with: 42.75 (in 7 real visits) roll(d2) : SOLVED with: 42.75 (in 7 real visits) roll(d1) : SOLVED with: 42.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.79198s. THTS: Maximal search depth set to 2 Search time: 0.018159s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 30 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 30 (in 2 real visits) roll(d3) : SOLVED with: 28.5 (in 7 real visits) roll(d2) : SOLVED with: 28.5 (in 7 real visits) roll(d1) : SOLVED with: 28.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 10/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.85384s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 36 Accumulated number of trials in root state: 1349030 Accumulated number of search nodes in root state: 4920056 Used RAM: 502744 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** >>> END OF ROUND 4 -- REWARD RECEIVED: 106 *********************************************** *********************************************** >>> STARTING ROUND 5 -- REMAINING TIME 238s *********************************************** *********************************************** Planning step 1/10 in round 5/10 Current state: | 0 0 0 Setting time for this decision to 3.91775s. THTS: Maximal search depth set to 10 Search time: 0.0181394s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 15 Exploration in Root: 4 Percentage Exploration in Root: 0.210526 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 5/10 Current state: | 0 5 0 Setting time for this decision to 3.98383s. THTS: Maximal search depth set to 9 Search time: 0.0181516s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 124.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 116.097 (in 2 real visits) roll(d3) : SOLVED with: 124.455 (in 7 real visits) roll(d2) : SOLVED with: 100.069 (in 7 real visits) roll(d1) : SOLVED with: 124.455 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 8 *********************************************** *********************************************** Planning step 3/10 in round 5/10 Current state: | 5 5 0 Setting time for this decision to 4.05221s. THTS: Maximal search depth set to 8 Search time: 0.0182016s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 129.87 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 125.556 (in 2 real visits) roll(d3) : SOLVED with: 129.87 (in 7 real visits) roll(d2) : SOLVED with: 113.097 (in 7 real visits) roll(d1) : SOLVED with: 113.097 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 13 *********************************************** *********************************************** Planning step 4/10 in round 5/10 Current state: | 5 5 2 Setting time for this decision to 4.12296s. THTS: Maximal search depth set to 7 Search time: 0.0181784s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 114.556 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 112.333 (in 2 real visits) roll(d3) : SOLVED with: 114.556 (in 7 real visits) roll(d2) : SOLVED with: 101.667 (in 7 real visits) roll(d1) : SOLVED with: 101.667 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 15 *********************************************** *********************************************** Planning step 5/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.19625s. THTS: Maximal search depth set to 6 Search time: 0.0185502s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 108 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 108 (in 2 real visits) roll(d3) : SOLVED with: 100.333 (in 7 real visits) roll(d2) : SOLVED with: 100.333 (in 7 real visits) roll(d1) : SOLVED with: 100.333 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 6/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.2722s. THTS: Maximal search depth set to 5 Search time: 0.0181444s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 90 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 90 (in 2 real visits) roll(d3) : SOLVED with: 83.25 (in 7 real visits) roll(d2) : SOLVED with: 83.25 (in 7 real visits) roll(d1) : SOLVED with: 83.25 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 7/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.35096s. THTS: Maximal search depth set to 4 Search time: 0.0181838s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 72 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 72 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 66.375 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 8/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.43272s. THTS: Maximal search depth set to 3 Search time: 0.0180297s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 54 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 54 (in 2 real visits) roll(d3) : SOLVED with: 49.75 (in 7 real visits) roll(d2) : SOLVED with: 49.75 (in 7 real visits) roll(d1) : SOLVED with: 49.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 9/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.5176s. THTS: Maximal search depth set to 2 Search time: 0.0179962s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 36 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 36 (in 2 real visits) roll(d3) : SOLVED with: 33.5 (in 7 real visits) roll(d2) : SOLVED with: 33.5 (in 7 real visits) roll(d1) : SOLVED with: 33.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 10/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.60582s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 46 Accumulated number of trials in root state: 1349049 Accumulated number of search nodes in root state: 4920080 Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** >>> END OF ROUND 5 -- REWARD RECEIVED: 147 *********************************************** *********************************************** >>> STARTING ROUND 6 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 6/10 Current state: | 0 0 0 Setting time for this decision to 4.69756s. THTS: Maximal search depth set to 10 Search time: 0.0180392s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 10 Exploration in Root: 9 Percentage Exploration in Root: 0.473684 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 6/10 Current state: | 0 0 4 Setting time for this decision to 4.79304s. THTS: Maximal search depth set to 9 Search time: 0.0183479s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 115.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 107.097 (in 2 real visits) roll(d3) : SOLVED with: 99.0692 (in 7 real visits) roll(d2) : SOLVED with: 115.455 (in 7 real visits) roll(d1) : SOLVED with: 115.455 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 7 *********************************************** *********************************************** Planning step 3/10 in round 6/10 Current state: | 2 0 4 Setting time for this decision to 4.89252s. THTS: Maximal search depth set to 8 Search time: 0.018055s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 104.796 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.6667 (in 2 real visits) roll(d3) : SOLVED with: 91.6673 (in 7 real visits) roll(d2) : SOLVED with: 104.796 (in 7 real visits) roll(d1) : SOLVED with: 102.097 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 9 *********************************************** *********************************************** Planning step 4/10 in round 6/10 Current state: | 2 1 4 Setting time for this decision to 4.99621s. THTS: Maximal search depth set to 7 Search time: 0.0181147s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 90.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.8333 (in 2 real visits) roll(d3) : SOLVED with: 79.5038 (in 7 real visits) roll(d2) : SOLVED with: 90.6667 (in 7 real visits) roll(d1) : SOLVED with: 89.1687 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 6/10 Current state: | 2 4 4 Setting time for this decision to 5.10441s. THTS: Maximal search depth set to 6 Search time: 0.0180585s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85.3333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.25 (in 2 real visits) roll(d3) : SOLVED with: 78.8333 (in 7 real visits) roll(d2) : SOLVED with: 78.8333 (in 7 real visits) roll(d1) : SOLVED with: 85.3333 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 6/10 in round 6/10 Current state: | 3 4 4 Setting time for this decision to 5.21744s. THTS: Maximal search depth set to 5 Search time: 0.0182129s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 71.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 70.375 (in 2 real visits) roll(d3) : SOLVED with: 67.375 (in 7 real visits) roll(d2) : SOLVED with: 67.375 (in 7 real visits) roll(d1) : SOLVED with: 71.25 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 14 *********************************************** *********************************************** Planning step 7/10 in round 6/10 Current state: | 2 4 4 Setting time for this decision to 5.33559s. THTS: Maximal search depth set to 4 Search time: 0.0181001s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 55.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 53.75 (in 2 real visits) roll(d3) : SOLVED with: 50.375 (in 7 real visits) roll(d2) : SOLVED with: 50.375 (in 7 real visits) roll(d1) : SOLVED with: 55.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 8/10 in round 6/10 Current state: | 1 4 4 Setting time for this decision to 5.45926s. THTS: Maximal search depth set to 3 Search time: 0.0181365s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 39.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 37.5 (in 2 real visits) roll(d3) : SOLVED with: 34.6667 (in 7 real visits) roll(d2) : SOLVED with: 34.6667 (in 7 real visits) roll(d1) : SOLVED with: 39.75 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 9/10 in round 6/10 Current state: | 5 4 4 Setting time for this decision to 5.58879s. THTS: Maximal search depth set to 2 Search time: 0.0181599s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 30.5 (in 7 real visits) roll(d2) : SOLVED with: 30.5 (in 7 real visits) roll(d1) : SOLVED with: 29.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 6/10 Current state: | 5 4 4 Setting time for this decision to 5.72463s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 56 Accumulated number of trials in root state: 1349068 Accumulated number of search nodes in root state: 4920104 Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 6 -- REWARD RECEIVED: 113 *********************************************** *********************************************** >>> STARTING ROUND 7 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 7/10 Current state: | 0 0 0 Setting time for this decision to 5.86728s. THTS: Maximal search depth set to 10 Search time: 0.0181742s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 11 Exploration in Root: 8 Percentage Exploration in Root: 0.421053 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 7/10 Current state: | 2 0 0 Setting time for this decision to 6.01726s. THTS: Maximal search depth set to 9 Search time: 0.0181986s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 102.465 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 92.6673 (in 2 real visits) roll(d3) : SOLVED with: 102.465 (in 7 real visits) roll(d2) : SOLVED with: 102.465 (in 7 real visits) roll(d1) : SOLVED with: 97.0692 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 5 *********************************************** *********************************************** Planning step 3/10 in round 7/10 Current state: | 2 0 2 Setting time for this decision to 6.17511s. THTS: Maximal search depth set to 8 Search time: 0.0181892s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.3666 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.0017 (in 2 real visits) roll(d3) : SOLVED with: 89.6673 (in 7 real visits) roll(d2) : SOLVED with: 92.3666 (in 7 real visits) roll(d1) : SOLVED with: 89.6673 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 7/10 Current state: | 2 1 2 Setting time for this decision to 6.34151s. THTS: Maximal search depth set to 7 Search time: 0.0181217s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 79.0017 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 73.1562 (in 2 real visits) roll(d3) : SOLVED with: 77.5038 (in 7 real visits) roll(d2) : SOLVED with: 79.0017 (in 7 real visits) roll(d1) : SOLVED with: 77.5038 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 7/10 Current state: | 2 5 2 Setting time for this decision to 6.51714s. THTS: Maximal search depth set to 6 Search time: 0.0180176s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 82.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 79.375 (in 2 real visits) roll(d3) : SOLVED with: 82.8333 (in 7 real visits) roll(d2) : SOLVED with: 69.1562 (in 7 real visits) roll(d1) : SOLVED with: 82.8333 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 12 *********************************************** *********************************************** Planning step 6/10 in round 7/10 Current state: | 2 5 1 Setting time for this decision to 6.7028s. THTS: Maximal search depth set to 5 Search time: 0.0181495s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 62.375 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 53.456 (in 7 real visits) roll(d1) : SOLVED with: 64.8935 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 11 *********************************************** *********************************************** Planning step 7/10 in round 7/10 Current state: | 2 5 2 Setting time for this decision to 6.89941s. THTS: Maximal search depth set to 4 Search time: 0.0182531s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 52.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 50 (in 2 real visits) roll(d3) : SOLVED with: 52.375 (in 7 real visits) roll(d2) : SOLVED with: 43.5 (in 7 real visits) roll(d1) : SOLVED with: 52.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 12 *********************************************** *********************************************** Planning step 8/10 in round 7/10 Current state: | 2 5 5 Setting time for this decision to 7.10791s. THTS: Maximal search depth set to 3 Search time: 0.0181674s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 46.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45.5 (in 2 real visits) roll(d3) : SOLVED with: 41 (in 7 real visits) roll(d2) : SOLVED with: 41 (in 7 real visits) roll(d1) : SOLVED with: 46.75 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 7/10 Current state: | 0 5 5 Setting time for this decision to 7.32944s. THTS: Maximal search depth set to 2 Search time: 0.0180116s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 28.5 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 26 (in 2 real visits) roll(d3) : SOLVED with: 23.5 (in 7 real visits) roll(d2) : SOLVED with: 23.5 (in 7 real visits) roll(d1) : SOLVED with: 28.5 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 10/10 in round 7/10 Current state: | 5 5 5 Setting time for this decision to 7.56529s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 66 Accumulated number of trials in root state: 1349087 Accumulated number of search nodes in root state: 4920128 Used RAM: 502744 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** >>> END OF ROUND 7 -- REWARD RECEIVED: 104 *********************************************** *********************************************** >>> STARTING ROUND 8 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 8/10 Current state: | 0 0 0 Setting time for this decision to 7.81683s. THTS: Maximal search depth set to 10 Search time: 0.0177993s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 12 Exploration in Root: 7 Percentage Exploration in Root: 0.368421 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 8/10 Current state: | 3 0 0 Setting time for this decision to 8.08576s. THTS: Maximal search depth set to 9 Search time: 0.0180285s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 107.43 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.597 (in 2 real visits) roll(d3) : SOLVED with: 107.43 (in 7 real visits) roll(d2) : SOLVED with: 107.43 (in 7 real visits) roll(d1) : SOLVED with: 98.0692 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 6 *********************************************** *********************************************** Planning step 3/10 in round 8/10 Current state: | 3 0 0 Setting time for this decision to 8.37386s. THTS: Maximal search depth set to 8 Search time: 0.0179919s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.597 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 84.1632 (in 2 real visits) roll(d3) : SOLVED with: 92.597 (in 7 real visits) roll(d2) : SOLVED with: 92.597 (in 7 real visits) roll(d1) : SOLVED with: 83.2784 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 6 *********************************************** *********************************************** Planning step 4/10 in round 8/10 Current state: | 3 0 4 Setting time for this decision to 8.6833s. THTS: Maximal search depth set to 7 Search time: 0.018327s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.8333 (in 2 real visits) roll(d3) : SOLVED with: 82.1632 (in 7 real visits) roll(d2) : SOLVED with: 92.6667 (in 7 real visits) roll(d1) : SOLVED with: 87.9687 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 8/10 Current state: | 3 1 4 Setting time for this decision to 9.01658s. THTS: Maximal search depth set to 6 Search time: 0.0180159s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 78.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 75.375 (in 2 real visits) roll(d3) : SOLVED with: 70.3812 (in 7 real visits) roll(d2) : SOLVED with: 78.8333 (in 7 real visits) roll(d1) : SOLVED with: 75.3395 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 8/10 Current state: | 3 3 4 Setting time for this decision to 9.37648s. THTS: Maximal search depth set to 5 Search time: 0.018113s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.375 (in 2 real visits) roll(d3) : SOLVED with: 62.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 66.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 8/10 Current state: | 5 3 4 Setting time for this decision to 9.76638s. THTS: Maximal search depth set to 4 Search time: 0.0180259s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 60.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 60 (in 2 real visits) roll(d3) : SOLVED with: 57.375 (in 7 real visits) roll(d2) : SOLVED with: 60.375 (in 7 real visits) roll(d1) : SOLVED with: 54.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 8/10 Current state: | 5 2 4 Setting time for this decision to 10.1902s. THTS: Maximal search depth set to 3 Search time: 0.0186874s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 43.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 42.5 (in 2 real visits) roll(d3) : SOLVED with: 40 (in 7 real visits) roll(d2) : SOLVED with: 43.75 (in 7 real visits) roll(d1) : SOLVED with: 38 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 14 *********************************************** *********************************************** Planning step 9/10 in round 8/10 Current state: | 5 4 4 Setting time for this decision to 10.6525s. THTS: Maximal search depth set to 2 Search time: 0.0181043s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 30.5 (in 7 real visits) roll(d2) : SOLVED with: 30.5 (in 7 real visits) roll(d1) : SOLVED with: 29.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 8/10 Current state: | 5 4 4 Setting time for this decision to 11.1589s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 76 Accumulated number of trials in root state: 1349106 Accumulated number of search nodes in root state: 4920152 Used RAM: 502744 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 8 -- REWARD RECEIVED: 110 *********************************************** *********************************************** >>> STARTING ROUND 9 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 9/10 Current state: | 0 0 0 Setting time for this decision to 11.7159s. THTS: Maximal search depth set to 10 Search time: 0.0181039s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 12 Exploration in Root: 7 Percentage Exploration in Root: 0.368421 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 9/10 Current state: | 0 0 0 Setting time for this decision to 12.3315s. THTS: Maximal search depth set to 9 Search time: 0.0181684s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 2 real visits) roll(d3) : SOLVED with: 95.0692 (in 7 real visits) roll(d2) : SOLVED with: 95.0692 (in 7 real visits) roll(d1) : SOLVED with: 95.0692 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 9/10 Current state: | 0 5 0 Setting time for this decision to 13.0156s. THTS: Maximal search depth set to 8 Search time: 0.0179806s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 108.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 99.9687 (in 2 real visits) roll(d3) : SOLVED with: 108.097 (in 7 real visits) roll(d2) : SOLVED with: 85.2784 (in 7 real visits) roll(d1) : SOLVED with: 108.097 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 8 *********************************************** *********************************************** Planning step 4/10 in round 9/10 Current state: | 0 5 0 Setting time for this decision to 13.7801s. THTS: Maximal search depth set to 7 Search time: 0.0186178s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 91.9687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 84.1397 (in 2 real visits) roll(d3) : SOLVED with: 91.9687 (in 7 real visits) roll(d2) : SOLVED with: 70.9334 (in 7 real visits) roll(d1) : SOLVED with: 91.9687 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 9/10 Current state: | 4 5 0 Setting time for this decision to 14.6402s. THTS: Maximal search depth set to 6 Search time: 0.0181202s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 89.3333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.25 (in 2 real visits) roll(d3) : SOLVED with: 89.3333 (in 7 real visits) roll(d2) : SOLVED with: 75.1397 (in 7 real visits) roll(d1) : SOLVED with: 80.1397 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 12 *********************************************** *********************************************** Planning step 6/10 in round 9/10 Current state: | 4 5 5 Setting time for this decision to 15.6149s. THTS: Maximal search depth set to 5 Search time: 0.0181953s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85 (in 2 real visits) roll(d3) : SOLVED with: 78.25 (in 7 real visits) roll(d2) : SOLVED with: 78.25 (in 7 real visits) roll(d1) : SOLVED with: 82.25 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 7/10 in round 9/10 Current state: | 4 5 5 Setting time for this decision to 16.729s. THTS: Maximal search depth set to 4 Search time: 0.0182055s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 68 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68 (in 2 real visits) roll(d3) : SOLVED with: 62.375 (in 7 real visits) roll(d2) : SOLVED with: 62.375 (in 7 real visits) roll(d1) : SOLVED with: 65.375 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 8/10 in round 9/10 Current state: | 4 5 5 Setting time for this decision to 18.0144s. THTS: Maximal search depth set to 3 Search time: 0.0180344s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 51 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51 (in 2 real visits) roll(d3) : SOLVED with: 46.75 (in 7 real visits) roll(d2) : SOLVED with: 46.75 (in 7 real visits) roll(d1) : SOLVED with: 48.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 9/10 in round 9/10 Current state: | 4 5 5 Setting time for this decision to 19.514s. THTS: Maximal search depth set to 2 Search time: 0.0186548s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 34 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 34 (in 2 real visits) roll(d3) : SOLVED with: 31.5 (in 7 real visits) roll(d2) : SOLVED with: 31.5 (in 7 real visits) roll(d1) : SOLVED with: 32.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 10/10 in round 9/10 Current state: | 4 5 5 Setting time for this decision to 21.2863s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 86 Accumulated number of trials in root state: 1349125 Accumulated number of search nodes in root state: 4920176 Used RAM: 502744 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** >>> END OF ROUND 9 -- REWARD RECEIVED: 119 *********************************************** *********************************************** >>> STARTING ROUND 10 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 10/10 Current state: | 0 0 0 Setting time for this decision to 23.413s. THTS: Maximal search depth set to 10 Search time: 0.0179703s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 10 Exploration in Root: 9 Percentage Exploration in Root: 0.473684 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 10/10 Current state: | 0 0 0 Setting time for this decision to 26.0124s. THTS: Maximal search depth set to 9 Search time: 0.0179552s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 2 real visits) roll(d3) : SOLVED with: 95.0692 (in 7 real visits) roll(d2) : SOLVED with: 95.0692 (in 7 real visits) roll(d1) : SOLVED with: 95.0692 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 10/10 Current state: | 4 0 0 Setting time for this decision to 29.2616s. THTS: Maximal search depth set to 8 Search time: 0.0182553s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 91.9687 (in 2 real visits) roll(d3) : SOLVED with: 100.097 (in 7 real visits) roll(d2) : SOLVED with: 100.097 (in 7 real visits) roll(d1) : SOLVED with: 84.2784 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 10/10 Current state: | 4 0 3 Setting time for this decision to 33.4393s. THTS: Maximal search depth set to 7 Search time: 0.0181153s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.8333 (in 2 real visits) roll(d3) : SOLVED with: 87.9687 (in 7 real visits) roll(d2) : SOLVED with: 92.6667 (in 7 real visits) roll(d1) : SOLVED with: 82.1632 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 10/10 Current state: | 4 1 3 Setting time for this decision to 39.0093s. THTS: Maximal search depth set to 6 Search time: 0.0190218s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 78.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 75.375 (in 2 real visits) roll(d3) : SOLVED with: 75.3395 (in 7 real visits) roll(d2) : SOLVED with: 78.8333 (in 7 real visits) roll(d1) : SOLVED with: 70.3812 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d2) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 46.8074s. THTS: Maximal search depth set to 5 Search time: 0.0179726s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.375 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 10/10 Current state: | 4 3 1 Setting time for this decision to 58.5045s. THTS: Maximal search depth set to 4 Search time: 0.0182284s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 50.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 47.75 (in 2 real visits) roll(d3) : SOLVED with: 50.375 (in 7 real visits) roll(d2) : SOLVED with: 46.9306 (in 7 real visits) roll(d1) : SOLVED with: 43.9306 (in 7 real visits) Used RAM: 502744 Submitted action: roll(d3) Immediate reward: 11 *********************************************** *********************************************** Planning step 8/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 77.9997s. THTS: Maximal search depth set to 3 Search time: 0.0181632s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 39 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 39 (in 2 real visits) roll(d3) : SOLVED with: 38.75 (in 7 real visits) roll(d2) : SOLVED with: 38.75 (in 7 real visits) roll(d1) : SOLVED with: 36.75 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** Planning step 9/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 116.99s. THTS: Maximal search depth set to 2 Search time: 0.0181989s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 17466098 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 26 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 26 (in 2 real visits) roll(d3) : SOLVED with: 25.5 (in 7 real visits) roll(d2) : SOLVED with: 25.5 (in 7 real visits) roll(d1) : SOLVED with: 24.5 (in 7 real visits) Used RAM: 502744 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** Planning step 10/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 233.962s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 96 Accumulated number of trials in root state: 1349144 Accumulated number of search nodes in root state: 4920200 Used RAM: 502744 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** >>> END OF ROUND 10 -- REWARD RECEIVED: 97 *********************************************** *********************************************** Immediate rewards: Round 0: 3 3 5 10 11 11 15 16 16 16 = 106 Round 1: 3 3 3 7 8 13 17 17 17 17 = 105 Round 2: 3 6 7 12 13 14 12 15 15 15 = 112 Round 3: 3 7 7 8 9 12 15 15 15 15 = 106 Round 4: 3 8 13 15 18 18 18 18 18 18 = 147 Round 5: 3 7 9 10 13 14 13 12 16 16 = 113 Round 6: 3 5 7 8 12 11 12 15 13 18 = 104 Round 7: 3 6 6 10 11 13 15 14 16 16 = 110 Round 8: 3 3 8 8 12 17 17 17 17 17 = 119 Round 9: 3 3 7 10 11 13 11 13 13 13 = 97 >>> TOTAL REWARD: 1119 >>> AVERAGE REWARD: 111.9 *********************************************** PROST complete running time: 30.2891s