/home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpSPw50q /home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpSPw50q *************** RDDL-PARSER CALL WITH 600SEC *************** Parsing... Setting outcome pruning to 0.1 ...finished (0.000993967s). instantiating... Instantiating variables... ...finished (1.00136e-05) Instantiating CPFs... ...finished (4.1008e-05) Instantiating preconditions... ...finished (5.96046e-06) ...finished (6.60419e-05s). preprocessing... Preparing evaluatables... ...finished (3.48091e-05) Preparing actions... ...finished (1.50204e-05) Calculating CPF domain... ...finished (2.19345e-05) Finalizing evaluatables... ...finished (2.09808e-05) Computing determinization... ...finished (2.90871e-05) Determining task properties... ...finished (2.14577e-06) Preparing hash keys... ...finished (9.05991e-06) Precomputing evaluatables... ...finished (0.000135899) Calculating min and max reward... ...finished (1.90735e-06) ...finished (0.000299215s). analyzing task... Creating training set with 93 candidates. ...finished (0.00164413s). writing output for instance dice_game_demo_inst_mdp__1... ...finished (0.00129294s). writing transition relations to json file... ...finished (0.000323057s). total time: 0.00465202s RDDL-Parser took: 0.0121559s learning... THTS: learning... DD_Heuristic: learning [31s (0.125%)] with /home/linuxbuildslave/buildslaves/ailinux/ipc-prob-build-singularity-linux/build/tmpSPw50q/dice_game_demo_inst_mdp__1.json... Horizon: 10 Round to dezimal: 2 { "actions": { "noop": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==s2])))" }, "roll(d1) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==0] * (1) + [s0_primed==1] * (1) + [s0_primed==2] * (1) + [s0_primed==3] * (1) + [s0_primed==4] * (1) + [s0_primed==5] * (1))) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==s2])))" }, "roll(d2) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==0] * (1) + [s1_primed==1] * (1) + [s1_primed==2] * (1) + [s1_primed==3] * (1) + [s1_primed==4] * (1) + [s1_primed==5] * (1))) * (1 * ([s2_primed==s2])))" }, "roll(d3) ": { "Tc": "(0 - ([s0 == 0] + ([s0 == 1] * 2) + ([s0 == 2] * 3) + ([s0 == 3] * 4) + ([s0 == 4] * 5) + ([s0 == 5] * 6) + [s1 == 0] + ([s1 == 1] * 2) + ([s1 == 2] * 3) + ([s1 == 3] * 4) + ([s1 == 4] * 5) + ([s1 == 5] * 6) + [s2 == 0] + ([s2 == 1] * 2) + ([s2 == 2] * 3) + ([s2 == 3] * 4) + ([s2 == 4] * 5) + ([s2 == 5] * 6)))", "Tr": "((1 * ([s0_primed==s0])) * (1 * ([s1_primed==s1])) * (1 * ([s2_primed==0] * (1) + [s2_primed==1] * (1) + [s2_primed==2] * (1) + [s2_primed==3] * (1) + [s2_primed==4] * (1) + [s2_primed==5] * (1))))" } }, "goal_state": { "fake_goal": 1 }, "initial_state": { "fake_goal": 0, "s0": 0, "s1": 0, "s2": 0 }, "variables": { "fake_goal": { "domain": 2 }, "s0": { "domain": 6 }, "s1": { "domain": 6 }, "s2": { "domain": 6 } } } Original ordering: s0 s1 s2 fake_goal Build ast.....done! Compute fan-in...done! Fan-in ordering: fake_goal s0 s1 s2 [s0 : 1] [s1 : 2] [s2 : 3] [fake_goal : 0] Num variables: 4 => 10 [ incl. primed: 20 ] noop......overall time: 0.04 => Time left: 31.21s roll(d1) ......overall time: 0.05 => Time left: 31.2s roll(d2) ......overall time: 0.07 => Time left: 31.18s roll(d3) ......overall time: 0.08 => Time left: 31.17s Plan step 1/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.08 => Time left: 31.17s Plan step 2/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.08 => Time left: 31.17s Plan step 3/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 4/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 5/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.09 => Time left: 31.16s Plan step 6/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 7/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 8/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 9/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Plan step 10/10... ...worst value: -0 ...overall worst value: -0 ...overall time: 0.1 => Time left: 31.15s Completed layers: 11 Reset Det Task. ... finished THTS: ...finished ...finished (0.246464s). Final task: ----------------Actions--------------- Action fluents: roll(d1) roll(d2) roll(d3) --------------- Legal Action Combinations: noop() : Index : 0 Relevant preconditions: --------------- roll(d3) : Index : 1 Relevant preconditions: --------------- roll(d2) : Index : 2 Relevant preconditions: --------------- roll(d1) : Index : 3 Relevant preconditions: --------------- -----------------CPFs----------------- die-value(d1) HashIndex: 0, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d1) : 1 Formula: case roll(d1) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d1) Determinized formula: case roll(d1) then 0 case 1 then die-value(d1) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5 KleeneHashKeyBase: 1 -------------- die-value(d2) HashIndex: 1, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d2) : 1 Formula: case roll(d2) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d2) Determinized formula: case roll(d2) then 0 case 1 then die-value(d2) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 6, 2: 12, 3: 18, 4: 24, 5: 30 KleeneHashKeyBase: 63 -------------- die-value(d3) HashIndex: 2, probabilistic, caching in vectors, Kleene caching in vectors of size 126. Action Hash Key Map: roll(d3) : 1 Formula: case roll(d3) then Discrete( [0 : 0.166666666666667] [1 : 0.166666666666667] [2 : 0.166666666666667] [3 : 0.166666666666667] [4 : 0.166666666666667] [5 : 0.166666666666667] ) case 1 then die-value(d3) Determinized formula: case roll(d3) then 0 case 1 then die-value(d3) Domain: @1 @2 @3 @4 @5 @6 HashKeyBase: 0: 0, 1: 36, 2: 72, 3: 108, 4: 144, 5: 180 KleeneHashKeyBase: 3969 -------------- Reward CPF: Reward HashIndex: 3, deterministic, caching in vectors, Kleene caching in maps. Action Hash Key Map: Formula: (+ (== die-value(d1) 0) (* (== die-value(d1) 1) 2) (* (== die-value(d1) 2) 3) (* (== die-value(d1) 3) 4) (* (== die-value(d1) 4) 5) (* (== die-value(d1) 5) 6) (== die-value(d2) 0) (* (== die-value(d2) 1) 2) (* (== die-value(d2) 2) 3) (* (== die-value(d2) 3) 4) (* (== die-value(d2) 4) 5) (* (== die-value(d2) 5) 6) (== die-value(d3) 0) (* (== die-value(d3) 1) 2) (* (== die-value(d3) 2) 3) (* (== die-value(d3) 3) 4) (* (== die-value(d3) 4) 5) (* (== die-value(d3) 5) 6) ) Minimal reward: 3 Maximal reward: 18 Is action independent: 1 ------State Fluent Hash Key Map------- a change of probabilistic state fluent 0 influences variables 0 (2) 3 (1) a change of probabilistic state fluent 1 influences variables 1 (2) 3 (6) a change of probabilistic state fluent 2 influences variables 2 (2) 3 (36) a change of variable 0 influences variables in Kleene states 0 (2) 3 (1) a change of variable 1 influences variables in Kleene states 1 (2) 3 (63) a change of variable 2 influences variables in Kleene states 2 (2) 3 (3969) ---------Action Preconditions--------- ----------Initial State--------------- die-value(d1): 0 die-value(d2): 0 die-value(d3): 0 Remaining Steps: 10 StateHashKey: 0 Hashing of States is possible. Hashing of KleeneStates is possible. No reward locks detected in the training phase. This task contains unreasonable actions only in the determinization. The final reward is determined by applying NOOP. *********************************************** >>> STARTING ROUND 1 -- REMAINING TIME 249s *********************************************** *********************************************** Planning step 1/10 in round 1/10 Current state: | 0 0 0 Setting time for this decision to 2.46685s. THTS: Maximal search depth set to 10 Search time: 2.46685s Statistics of THTS: Performed trials: 567882 Created SearchNodes: 2839415 Cache Hits: 0 Action Selection: Exploitation in Root: 219341 Exploration in Root: 348541 Percentage Exploration in Root: 0.613756 Skipped backups: 812151531 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 117.336 (in 567886 real visits) Q-Value Estimates: noop() : 109.456 (in 3850 real visits) roll(d3) : 117.312 (in 183899 real visits) roll(d2) : 117.336 (in 192496 real visits) roll(d1) : 117.322 (in 187641 real visits) Used RAM: 489528 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 1/10 Current state: | 0 0 0 Setting time for this decision to 2.46684s. THTS: Maximal search depth set to 9 Search time: 2.46684s Statistics of THTS: Performed trials: 530094 Created SearchNodes: 2650311 Cache Hits: 0 Skipped backups: 815551193 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 99.3416 (in 530098 real visits) Q-Value Estimates: noop() : 91.8171 (in 3532 real visits) roll(d3) : 99.3355 (in 174386 real visits) roll(d2) : 99.3416 (in 176568 real visits) roll(d1) : 99.3387 (in 175612 real visits) Used RAM: 489528 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 1/10 Current state: | 0 2 0 Setting time for this decision to 2.46684s. THTS: Maximal search depth set to 8 Search time: 2.46684s Statistics of THTS: Performed trials: 526719 Created SearchNodes: 2553240 Cache Hits: 6869 Skipped backups: 819051951 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 88.4926 (in 526723 real visits) Q-Value Estimates: noop() : 82.4292 (in 5103 real visits) roll(d3) : 88.4926 (in 255104 real visits) roll(d2) : 86.5946 (in 16086 real visits) roll(d1) : 88.4851 (in 250430 real visits) Used RAM: 489528 Submitted action: roll(d3) Immediate reward: 5 *********************************************** *********************************************** Planning step 4/10 in round 1/10 Current state: | 0 2 5 Setting time for this decision to 2.46682s. THTS: Maximal search depth set to 7 Search time: 0.917916s Statistics of THTS: Performed trials: 234436 Created SearchNodes: 536171 Cache Hits: 106874 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 96.6667 (in 234440 real visits) Q-Value Estimates: noop() : SOLVED with: 90.8333 (in 4222 real visits) roll(d3) : SOLVED with: 78.3038 (in 4311 real visits) roll(d2) : SOLVED with: 93.9687 (in 14858 real visits) roll(d1) : SOLVED with: 96.6667 (in 211049 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 1/10 Current state: | 1 2 5 Setting time for this decision to 2.48296s. THTS: Maximal search depth set to 6 Search time: 0.0340782s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 81.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 77.375 (in 2 real visits) roll(d3) : SOLVED with: 66.6624 (in 7 real visits) roll(d2) : SOLVED with: 80.3395 (in 7 real visits) roll(d1) : SOLVED with: 81.8333 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 1/10 Current state: | 1 2 5 Setting time for this decision to 2.50873s. THTS: Maximal search depth set to 5 Search time: 0.0170826s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 62.375 (in 2 real visits) roll(d3) : SOLVED with: 53.456 (in 7 real visits) roll(d2) : SOLVED with: 64.8935 (in 7 real visits) roll(d1) : SOLVED with: 66.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 11 *********************************************** *********************************************** Planning step 7/10 in round 1/10 Current state: | 5 2 5 Setting time for this decision to 2.53523s. THTS: Maximal search depth set to 4 Search time: 0.0170785s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 63.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 61.75 (in 2 real visits) roll(d3) : SOLVED with: 55.375 (in 7 real visits) roll(d2) : SOLVED with: 63.375 (in 7 real visits) roll(d1) : SOLVED with: 55.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 1/10 Current state: | 5 3 5 Setting time for this decision to 2.5623s. THTS: Maximal search depth set to 3 Search time: 0.0172228s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 48 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 48 (in 2 real visits) roll(d3) : SOLVED with: 43.75 (in 7 real visits) roll(d2) : SOLVED with: 47.75 (in 7 real visits) roll(d1) : SOLVED with: 43.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 9/10 in round 1/10 Current state: | 5 3 5 Setting time for this decision to 2.58996s. THTS: Maximal search depth set to 2 Search time: 0.0171996s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 820617503 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 29.5 (in 7 real visits) roll(d2) : SOLVED with: 31.5 (in 7 real visits) roll(d1) : SOLVED with: 29.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 1/10 Current state: | 5 3 5 Setting time for this decision to 2.61822s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 7 Accumulated number of trials in root state: 567882 Accumulated number of search nodes in root state: 2839415 Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 1 -- REWARD RECEIVED: 106 *********************************************** *********************************************** >>> STARTING ROUND 2 -- REMAINING TIME 241s *********************************************** *********************************************** Planning step 1/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.64711s. THTS: Maximal search depth set to 10 Search time: 2.64712s Statistics of THTS: Performed trials: 722829 Created SearchNodes: 1927043 Cache Hits: 407560 Action Selection: Exploitation in Root: 253624 Exploration in Root: 469205 Percentage Exploration in Root: 0.649123 Skipped backups: 825199749 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: 113.2 (in 722833 real visits) Q-Value Estimates: noop() : 108.128 (in 5024 real visits) roll(d3) : 113.18 (in 236130 real visits) roll(d2) : 113.179 (in 234952 real visits) roll(d1) : 113.2 (in 246727 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.64711s. THTS: Maximal search depth set to 9 Search time: 0.130149s Statistics of THTS: Performed trials: 22981 Created SearchNodes: 36662 Cache Hits: 18902 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 22985 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 562 real visits) roll(d3) : SOLVED with: 95.0692 (in 7860 real visits) roll(d2) : SOLVED with: 95.0692 (in 6769 real visits) roll(d1) : SOLVED with: 95.0692 (in 7794 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 2/10 Current state: | 0 0 0 Setting time for this decision to 2.6757s. THTS: Maximal search depth set to 8 Search time: 0.0186324s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 80.2784 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68.9334 (in 2 real visits) roll(d3) : SOLVED with: 80.2784 (in 7 real visits) roll(d2) : SOLVED with: 80.2784 (in 7 real visits) roll(d1) : SOLVED with: 80.2784 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 4/10 in round 2/10 Current state: | 0 0 4 Setting time for this decision to 2.70624s. THTS: Maximal search depth set to 7 Search time: 0.0174704s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 84.9687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 77.1397 (in 2 real visits) roll(d3) : SOLVED with: 69.9334 (in 7 real visits) roll(d2) : SOLVED with: 84.9687 (in 7 real visits) roll(d1) : SOLVED with: 84.9687 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 7 *********************************************** *********************************************** Planning step 5/10 in round 2/10 Current state: | 0 1 4 Setting time for this decision to 2.7375s. THTS: Maximal search depth set to 6 Search time: 0.0179115s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 72.3395 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.8935 (in 2 real visits) roll(d3) : SOLVED with: 59.5367 (in 7 real visits) roll(d2) : SOLVED with: 71.1397 (in 7 real visits) roll(d1) : SOLVED with: 72.3395 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 8 *********************************************** *********************************************** Planning step 6/10 in round 2/10 Current state: | 5 1 4 Setting time for this decision to 2.76948s. THTS: Maximal search depth set to 5 Search time: 0.017159s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 74.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 71.375 (in 2 real visits) roll(d3) : SOLVED with: 66.8935 (in 7 real visits) roll(d2) : SOLVED with: 74.25 (in 7 real visits) roll(d1) : SOLVED with: 62.8935 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.80225s. THTS: Maximal search depth set to 4 Search time: 0.0175775s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 68 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68 (in 2 real visits) roll(d3) : SOLVED with: 65.375 (in 7 real visits) roll(d2) : SOLVED with: 62.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 8/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.83578s. THTS: Maximal search depth set to 3 Search time: 0.0173783s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 51 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51 (in 2 real visits) roll(d3) : SOLVED with: 48.75 (in 7 real visits) roll(d2) : SOLVED with: 46.75 (in 7 real visits) roll(d1) : SOLVED with: 46.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 9/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.87016s. THTS: Maximal search depth set to 2 Search time: 0.01733s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825286401 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 34 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 34 (in 2 real visits) roll(d3) : SOLVED with: 32.5 (in 7 real visits) roll(d2) : SOLVED with: 31.5 (in 7 real visits) roll(d1) : SOLVED with: 31.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 10/10 in round 2/10 Current state: | 5 5 4 Setting time for this decision to 2.90537s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 16 Accumulated number of trials in root state: 1290711 Accumulated number of search nodes in root state: 4766458 Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** >>> END OF ROUND 2 -- REWARD RECEIVED: 105 *********************************************** *********************************************** >>> STARTING ROUND 3 -- REMAINING TIME 238s *********************************************** *********************************************** Planning step 1/10 in round 3/10 Current state: | 0 0 0 Setting time for this decision to 2.94146s. THTS: Maximal search depth set to 10 Search time: 0.0406277s Statistics of THTS: Performed trials: 12612 Created SearchNodes: 15529 Cache Hits: 11751 Action Selection: Exploitation in Root: 4845 Exploration in Root: 7767 Percentage Exploration in Root: 0.615842 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 12616 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 4238 real visits) roll(d2) : SOLVED with: 110.224 (in 4207 real visits) roll(d1) : SOLVED with: 110.224 (in 4169 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 3/10 Current state: | 3 0 0 Setting time for this decision to 2.97818s. THTS: Maximal search depth set to 9 Search time: 0.018018s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 107.43 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.597 (in 2 real visits) roll(d3) : SOLVED with: 107.43 (in 7 real visits) roll(d2) : SOLVED with: 107.43 (in 7 real visits) roll(d1) : SOLVED with: 98.0692 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 6 *********************************************** *********************************************** Planning step 3/10 in round 3/10 Current state: | 3 1 0 Setting time for this decision to 3.01612s. THTS: Maximal search depth set to 8 Search time: 0.0175495s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 94.797 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.3632 (in 2 real visits) roll(d3) : SOLVED with: 94.797 (in 7 real visits) roll(d2) : SOLVED with: 93.597 (in 7 real visits) roll(d1) : SOLVED with: 86.6783 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 3/10 Current state: | 3 1 5 Setting time for this decision to 3.05505s. THTS: Maximal search depth set to 7 Search time: 0.0173023s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 96.8333 (in 2 real visits) roll(d3) : SOLVED with: 85.3632 (in 7 real visits) roll(d2) : SOLVED with: 100.667 (in 7 real visits) roll(d1) : SOLVED with: 97.1687 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 12 *********************************************** *********************************************** Planning step 5/10 in round 3/10 Current state: | 3 2 5 Setting time for this decision to 3.09501s. THTS: Maximal search depth set to 6 Search time: 0.0171231s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.375 (in 2 real visits) roll(d3) : SOLVED with: 73.875 (in 7 real visits) roll(d2) : SOLVED with: 85.8333 (in 7 real visits) roll(d1) : SOLVED with: 83.8333 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 13 *********************************************** *********************************************** Planning step 6/10 in round 3/10 Current state: | 3 3 5 Setting time for this decision to 3.13605s. THTS: Maximal search depth set to 5 Search time: 0.017247s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 71.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 70.375 (in 2 real visits) roll(d3) : SOLVED with: 63.375 (in 7 real visits) roll(d2) : SOLVED with: 71.375 (in 7 real visits) roll(d1) : SOLVED with: 71.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 14 *********************************************** *********************************************** Planning step 7/10 in round 3/10 Current state: | 1 3 5 Setting time for this decision to 3.17819s. THTS: Maximal search depth set to 4 Search time: 0.0171253s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 54.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51.75 (in 2 real visits) roll(d3) : SOLVED with: 44.9306 (in 7 real visits) roll(d2) : SOLVED with: 50.9306 (in 7 real visits) roll(d1) : SOLVED with: 54.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 8/10 in round 3/10 Current state: | 4 3 5 Setting time for this decision to 3.22148s. THTS: Maximal search depth set to 3 Search time: 0.017207s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 45 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45 (in 2 real visits) roll(d3) : SOLVED with: 40.75 (in 7 real visits) roll(d2) : SOLVED with: 44.75 (in 7 real visits) roll(d1) : SOLVED with: 42.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 3/10 Current state: | 4 3 5 Setting time for this decision to 3.26599s. THTS: Maximal search depth set to 2 Search time: 0.0169062s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 30 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 30 (in 2 real visits) roll(d3) : SOLVED with: 27.5 (in 7 real visits) roll(d2) : SOLVED with: 29.5 (in 7 real visits) roll(d1) : SOLVED with: 28.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 10/10 in round 3/10 Current state: | 4 3 5 Setting time for this decision to 3.31173s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 26 Accumulated number of trials in root state: 1303323 Accumulated number of search nodes in root state: 4781987 Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** >>> END OF ROUND 3 -- REWARD RECEIVED: 112 *********************************************** *********************************************** >>> STARTING ROUND 4 -- REMAINING TIME 238s *********************************************** *********************************************** Planning step 1/10 in round 4/10 Current state: | 0 0 0 Setting time for this decision to 3.35879s. THTS: Maximal search depth set to 10 Search time: 0.0173233s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 11 Exploration in Root: 8 Percentage Exploration in Root: 0.421053 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 4/10 Current state: | 0 4 0 Setting time for this decision to 3.4072s. THTS: Maximal search depth set to 9 Search time: 0.0181422s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 115.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 107.097 (in 2 real visits) roll(d3) : SOLVED with: 115.455 (in 7 real visits) roll(d2) : SOLVED with: 99.0692 (in 7 real visits) roll(d1) : SOLVED with: 115.455 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 7 *********************************************** *********************************************** Planning step 3/10 in round 4/10 Current state: | 0 4 0 Setting time for this decision to 3.45703s. THTS: Maximal search depth set to 8 Search time: 0.0172509s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 91.9687 (in 2 real visits) roll(d3) : SOLVED with: 100.097 (in 7 real visits) roll(d2) : SOLVED with: 84.2784 (in 7 real visits) roll(d1) : SOLVED with: 100.097 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 4/10 Current state: | 1 4 0 Setting time for this decision to 3.50837s. THTS: Maximal search depth set to 7 Search time: 0.0172745s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 87.1687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 80.3395 (in 2 real visits) roll(d3) : SOLVED with: 87.1687 (in 7 real visits) roll(d2) : SOLVED with: 73.3326 (in 7 real visits) roll(d1) : SOLVED with: 85.9688 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 4/10 Current state: | 1 4 1 Setting time for this decision to 3.56126s. THTS: Maximal search depth set to 6 Search time: 0.0170095s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 73.3395 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 67.8935 (in 2 real visits) roll(d3) : SOLVED with: 73.3395 (in 7 real visits) roll(d2) : SOLVED with: 61.7365 (in 7 real visits) roll(d1) : SOLVED with: 73.3395 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 9 *********************************************** *********************************************** Planning step 6/10 in round 4/10 Current state: | 1 4 4 Setting time for this decision to 3.61577s. THTS: Maximal search depth set to 5 Search time: 0.0170574s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 69.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 66.375 (in 2 real visits) roll(d3) : SOLVED with: 61.8935 (in 7 real visits) roll(d2) : SOLVED with: 61.8935 (in 7 real visits) roll(d1) : SOLVED with: 69.25 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 7/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.672s. THTS: Maximal search depth set to 4 Search time: 0.0174455s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 60 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 60 (in 2 real visits) roll(d3) : SOLVED with: 57.375 (in 7 real visits) roll(d2) : SOLVED with: 57.375 (in 7 real visits) roll(d1) : SOLVED with: 57.375 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.73s. THTS: Maximal search depth set to 3 Search time: 0.0170428s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 45 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45 (in 2 real visits) roll(d3) : SOLVED with: 42.75 (in 7 real visits) roll(d2) : SOLVED with: 42.75 (in 7 real visits) roll(d1) : SOLVED with: 42.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.78987s. THTS: Maximal search depth set to 2 Search time: 0.0172208s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 30 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 30 (in 2 real visits) roll(d3) : SOLVED with: 28.5 (in 7 real visits) roll(d2) : SOLVED with: 28.5 (in 7 real visits) roll(d1) : SOLVED with: 28.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** Planning step 10/10 in round 4/10 Current state: | 4 4 4 Setting time for this decision to 3.8517s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 36 Accumulated number of trials in root state: 1303342 Accumulated number of search nodes in root state: 4782011 Used RAM: 489572 Submitted action: noop() Immediate reward: 15 *********************************************** *********************************************** >>> END OF ROUND 4 -- REWARD RECEIVED: 106 *********************************************** *********************************************** >>> STARTING ROUND 5 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 5/10 Current state: | 0 0 0 Setting time for this decision to 3.9156s. THTS: Maximal search depth set to 10 Search time: 0.0171726s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 13 Exploration in Root: 6 Percentage Exploration in Root: 0.315789 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 5/10 Current state: | 5 0 0 Setting time for this decision to 3.98168s. THTS: Maximal search depth set to 9 Search time: 0.0172669s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 124.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 116.097 (in 2 real visits) roll(d3) : SOLVED with: 124.455 (in 7 real visits) roll(d2) : SOLVED with: 124.455 (in 7 real visits) roll(d1) : SOLVED with: 100.069 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 8 *********************************************** *********************************************** Planning step 3/10 in round 5/10 Current state: | 5 0 5 Setting time for this decision to 4.05002s. THTS: Maximal search depth set to 8 Search time: 0.0172704s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 129.87 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 125.556 (in 2 real visits) roll(d3) : SOLVED with: 113.097 (in 7 real visits) roll(d2) : SOLVED with: 129.87 (in 7 real visits) roll(d1) : SOLVED with: 113.097 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 13 *********************************************** *********************************************** Planning step 4/10 in round 5/10 Current state: | 5 2 5 Setting time for this decision to 4.12075s. THTS: Maximal search depth set to 7 Search time: 0.0172253s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 114.556 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 112.333 (in 2 real visits) roll(d3) : SOLVED with: 101.667 (in 7 real visits) roll(d2) : SOLVED with: 114.556 (in 7 real visits) roll(d1) : SOLVED with: 101.667 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 15 *********************************************** *********************************************** Planning step 5/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.19402s. THTS: Maximal search depth set to 6 Search time: 0.0187594s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 108 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 108 (in 2 real visits) roll(d3) : SOLVED with: 100.333 (in 7 real visits) roll(d2) : SOLVED with: 100.333 (in 7 real visits) roll(d1) : SOLVED with: 100.333 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 6/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.26993s. THTS: Maximal search depth set to 5 Search time: 0.016972s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 90 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 90 (in 2 real visits) roll(d3) : SOLVED with: 83.25 (in 7 real visits) roll(d2) : SOLVED with: 83.25 (in 7 real visits) roll(d1) : SOLVED with: 83.25 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 7/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.34867s. THTS: Maximal search depth set to 4 Search time: 0.0173148s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 72 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 72 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 66.375 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 8/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.4304s. THTS: Maximal search depth set to 3 Search time: 0.0172841s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 54 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 54 (in 2 real visits) roll(d3) : SOLVED with: 49.75 (in 7 real visits) roll(d2) : SOLVED with: 49.75 (in 7 real visits) roll(d1) : SOLVED with: 49.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 9/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.51525s. THTS: Maximal search depth set to 2 Search time: 0.0171322s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 36 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 36 (in 2 real visits) roll(d3) : SOLVED with: 33.5 (in 7 real visits) roll(d2) : SOLVED with: 33.5 (in 7 real visits) roll(d1) : SOLVED with: 33.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** Planning step 10/10 in round 5/10 Current state: | 5 5 5 Setting time for this decision to 4.60343s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 46 Accumulated number of trials in root state: 1303361 Accumulated number of search nodes in root state: 4782035 Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** >>> END OF ROUND 5 -- REWARD RECEIVED: 147 *********************************************** *********************************************** >>> STARTING ROUND 6 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 6/10 Current state: | 0 0 0 Setting time for this decision to 4.69514s. THTS: Maximal search depth set to 10 Search time: 0.0172439s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 11 Exploration in Root: 8 Percentage Exploration in Root: 0.421053 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 6/10 Current state: | 0 4 0 Setting time for this decision to 4.79059s. THTS: Maximal search depth set to 9 Search time: 0.0173548s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 115.455 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 107.097 (in 2 real visits) roll(d3) : SOLVED with: 115.455 (in 7 real visits) roll(d2) : SOLVED with: 99.0692 (in 7 real visits) roll(d1) : SOLVED with: 115.455 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 7 *********************************************** *********************************************** Planning step 3/10 in round 6/10 Current state: | 2 4 0 Setting time for this decision to 4.89004s. THTS: Maximal search depth set to 8 Search time: 0.0171978s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 104.796 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.6667 (in 2 real visits) roll(d3) : SOLVED with: 104.796 (in 7 real visits) roll(d2) : SOLVED with: 91.6673 (in 7 real visits) roll(d1) : SOLVED with: 102.097 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 9 *********************************************** *********************************************** Planning step 4/10 in round 6/10 Current state: | 2 4 1 Setting time for this decision to 4.9937s. THTS: Maximal search depth set to 7 Search time: 0.0171328s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 90.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.8333 (in 2 real visits) roll(d3) : SOLVED with: 90.6667 (in 7 real visits) roll(d2) : SOLVED with: 79.5038 (in 7 real visits) roll(d1) : SOLVED with: 89.1687 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 6/10 Current state: | 2 4 4 Setting time for this decision to 5.10187s. THTS: Maximal search depth set to 6 Search time: 0.017105s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85.3333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.25 (in 2 real visits) roll(d3) : SOLVED with: 78.8333 (in 7 real visits) roll(d2) : SOLVED with: 78.8333 (in 7 real visits) roll(d1) : SOLVED with: 85.3333 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 6/10 in round 6/10 Current state: | 3 4 4 Setting time for this decision to 5.21484s. THTS: Maximal search depth set to 5 Search time: 0.0197091s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 71.25 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 70.375 (in 2 real visits) roll(d3) : SOLVED with: 67.375 (in 7 real visits) roll(d2) : SOLVED with: 67.375 (in 7 real visits) roll(d1) : SOLVED with: 71.25 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 14 *********************************************** *********************************************** Planning step 7/10 in round 6/10 Current state: | 2 4 4 Setting time for this decision to 5.33291s. THTS: Maximal search depth set to 4 Search time: 0.0172435s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 55.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 53.75 (in 2 real visits) roll(d3) : SOLVED with: 50.375 (in 7 real visits) roll(d2) : SOLVED with: 50.375 (in 7 real visits) roll(d1) : SOLVED with: 55.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 13 *********************************************** *********************************************** Planning step 8/10 in round 6/10 Current state: | 1 4 4 Setting time for this decision to 5.45651s. THTS: Maximal search depth set to 3 Search time: 0.0171713s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 39.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 37.5 (in 2 real visits) roll(d3) : SOLVED with: 34.6667 (in 7 real visits) roll(d2) : SOLVED with: 34.6667 (in 7 real visits) roll(d1) : SOLVED with: 39.75 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 9/10 in round 6/10 Current state: | 5 4 4 Setting time for this decision to 5.58602s. THTS: Maximal search depth set to 2 Search time: 0.0172195s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 30.5 (in 7 real visits) roll(d2) : SOLVED with: 30.5 (in 7 real visits) roll(d1) : SOLVED with: 29.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 6/10 Current state: | 5 4 4 Setting time for this decision to 5.72183s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 56 Accumulated number of trials in root state: 1303380 Accumulated number of search nodes in root state: 4782059 Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 6 -- REWARD RECEIVED: 113 *********************************************** *********************************************** >>> STARTING ROUND 7 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 7/10 Current state: | 0 0 0 Setting time for this decision to 5.86442s. THTS: Maximal search depth set to 10 Search time: 0.0171407s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 10 Exploration in Root: 9 Percentage Exploration in Root: 0.473684 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 7/10 Current state: | 2 0 0 Setting time for this decision to 6.01433s. THTS: Maximal search depth set to 9 Search time: 0.0170329s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 102.465 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 92.6673 (in 2 real visits) roll(d3) : SOLVED with: 102.465 (in 7 real visits) roll(d2) : SOLVED with: 102.465 (in 7 real visits) roll(d1) : SOLVED with: 97.0692 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 5 *********************************************** *********************************************** Planning step 3/10 in round 7/10 Current state: | 2 0 2 Setting time for this decision to 6.17216s. THTS: Maximal search depth set to 8 Search time: 0.0170222s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.3666 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.0017 (in 2 real visits) roll(d3) : SOLVED with: 89.6673 (in 7 real visits) roll(d2) : SOLVED with: 92.3666 (in 7 real visits) roll(d1) : SOLVED with: 89.6673 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 7/10 Current state: | 2 1 2 Setting time for this decision to 6.33849s. THTS: Maximal search depth set to 7 Search time: 0.017327s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 79.0017 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 73.1562 (in 2 real visits) roll(d3) : SOLVED with: 77.5038 (in 7 real visits) roll(d2) : SOLVED with: 79.0017 (in 7 real visits) roll(d1) : SOLVED with: 77.5038 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 7/10 Current state: | 2 5 2 Setting time for this decision to 6.51406s. THTS: Maximal search depth set to 6 Search time: 0.0182074s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 82.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 79.375 (in 2 real visits) roll(d3) : SOLVED with: 82.8333 (in 7 real visits) roll(d2) : SOLVED with: 69.1562 (in 7 real visits) roll(d1) : SOLVED with: 82.8333 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 6/10 in round 7/10 Current state: | 1 5 2 Setting time for this decision to 6.69963s. THTS: Maximal search depth set to 5 Search time: 0.0170486s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 62.375 (in 2 real visits) roll(d3) : SOLVED with: 64.8935 (in 7 real visits) roll(d2) : SOLVED with: 53.456 (in 7 real visits) roll(d1) : SOLVED with: 66.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 11 *********************************************** *********************************************** Planning step 7/10 in round 7/10 Current state: | 2 5 2 Setting time for this decision to 6.89618s. THTS: Maximal search depth set to 4 Search time: 0.0170863s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 52.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 50 (in 2 real visits) roll(d3) : SOLVED with: 52.375 (in 7 real visits) roll(d2) : SOLVED with: 43.5 (in 7 real visits) roll(d1) : SOLVED with: 52.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 8/10 in round 7/10 Current state: | 5 5 2 Setting time for this decision to 7.10461s. THTS: Maximal search depth set to 3 Search time: 0.0168508s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 46.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 45.5 (in 2 real visits) roll(d3) : SOLVED with: 46.75 (in 7 real visits) roll(d2) : SOLVED with: 41 (in 7 real visits) roll(d1) : SOLVED with: 41 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 15 *********************************************** *********************************************** Planning step 9/10 in round 7/10 Current state: | 5 5 0 Setting time for this decision to 7.32609s. THTS: Maximal search depth set to 2 Search time: 0.0171967s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 28.5 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 26 (in 2 real visits) roll(d3) : SOLVED with: 28.5 (in 7 real visits) roll(d2) : SOLVED with: 23.5 (in 7 real visits) roll(d1) : SOLVED with: 23.5 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 13 *********************************************** *********************************************** Planning step 10/10 in round 7/10 Current state: | 5 5 5 Setting time for this decision to 7.56184s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 66 Accumulated number of trials in root state: 1303399 Accumulated number of search nodes in root state: 4782083 Used RAM: 489572 Submitted action: noop() Immediate reward: 18 *********************************************** *********************************************** >>> END OF ROUND 7 -- REWARD RECEIVED: 104 *********************************************** *********************************************** >>> STARTING ROUND 8 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 8/10 Current state: | 0 0 0 Setting time for this decision to 7.8133s. THTS: Maximal search depth set to 10 Search time: 0.0173094s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 12 Exploration in Root: 7 Percentage Exploration in Root: 0.368421 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 8/10 Current state: | 0 3 0 Setting time for this decision to 8.0821s. THTS: Maximal search depth set to 9 Search time: 0.0173367s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 107.43 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.597 (in 2 real visits) roll(d3) : SOLVED with: 107.43 (in 7 real visits) roll(d2) : SOLVED with: 98.0692 (in 7 real visits) roll(d1) : SOLVED with: 107.43 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 6 *********************************************** *********************************************** Planning step 3/10 in round 8/10 Current state: | 0 3 0 Setting time for this decision to 8.37014s. THTS: Maximal search depth set to 8 Search time: 0.017309s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.597 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 84.1632 (in 2 real visits) roll(d3) : SOLVED with: 92.597 (in 7 real visits) roll(d2) : SOLVED with: 83.2784 (in 7 real visits) roll(d1) : SOLVED with: 92.597 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 6 *********************************************** *********************************************** Planning step 4/10 in round 8/10 Current state: | 4 3 0 Setting time for this decision to 8.67948s. THTS: Maximal search depth set to 7 Search time: 0.0175706s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.8333 (in 2 real visits) roll(d3) : SOLVED with: 92.6667 (in 7 real visits) roll(d2) : SOLVED with: 87.9687 (in 7 real visits) roll(d1) : SOLVED with: 82.1632 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 8/10 Current state: | 4 3 1 Setting time for this decision to 9.01262s. THTS: Maximal search depth set to 6 Search time: 0.0170195s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 78.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 75.375 (in 2 real visits) roll(d3) : SOLVED with: 78.8333 (in 7 real visits) roll(d2) : SOLVED with: 75.3395 (in 7 real visits) roll(d1) : SOLVED with: 70.3812 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 8/10 Current state: | 4 3 3 Setting time for this decision to 9.3724s. THTS: Maximal search depth set to 5 Search time: 0.0171538s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.375 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 8/10 Current state: | 4 5 3 Setting time for this decision to 9.76221s. THTS: Maximal search depth set to 4 Search time: 0.0169751s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 60.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 60 (in 2 real visits) roll(d3) : SOLVED with: 60.375 (in 7 real visits) roll(d2) : SOLVED with: 54.375 (in 7 real visits) roll(d1) : SOLVED with: 57.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 15 *********************************************** *********************************************** Planning step 8/10 in round 8/10 Current state: | 4 5 2 Setting time for this decision to 10.1859s. THTS: Maximal search depth set to 3 Search time: 0.0173186s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 43.75 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 42.5 (in 2 real visits) roll(d3) : SOLVED with: 43.75 (in 7 real visits) roll(d2) : SOLVED with: 38 (in 7 real visits) roll(d1) : SOLVED with: 40 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 14 *********************************************** *********************************************** Planning step 9/10 in round 8/10 Current state: | 4 5 4 Setting time for this decision to 10.648s. THTS: Maximal search depth set to 2 Search time: 0.0173164s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 32 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 32 (in 2 real visits) roll(d3) : SOLVED with: 30.5 (in 7 real visits) roll(d2) : SOLVED with: 29.5 (in 7 real visits) roll(d1) : SOLVED with: 30.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** Planning step 10/10 in round 8/10 Current state: | 4 5 4 Setting time for this decision to 11.1543s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 76 Accumulated number of trials in root state: 1303418 Accumulated number of search nodes in root state: 4782107 Used RAM: 489572 Submitted action: noop() Immediate reward: 16 *********************************************** *********************************************** >>> END OF ROUND 8 -- REWARD RECEIVED: 110 *********************************************** *********************************************** >>> STARTING ROUND 9 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 9/10 Current state: | 0 0 0 Setting time for this decision to 11.7111s. THTS: Maximal search depth set to 10 Search time: 0.017172s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 11 Exploration in Root: 8 Percentage Exploration in Root: 0.421053 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 9/10 Current state: | 0 0 0 Setting time for this decision to 12.3265s. THTS: Maximal search depth set to 9 Search time: 0.0170835s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 2 real visits) roll(d3) : SOLVED with: 95.0692 (in 7 real visits) roll(d2) : SOLVED with: 95.0692 (in 7 real visits) roll(d1) : SOLVED with: 95.0692 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 9/10 Current state: | 0 5 0 Setting time for this decision to 13.0103s. THTS: Maximal search depth set to 8 Search time: 0.0171081s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 108.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 99.9687 (in 2 real visits) roll(d3) : SOLVED with: 108.097 (in 7 real visits) roll(d2) : SOLVED with: 85.2784 (in 7 real visits) roll(d1) : SOLVED with: 108.097 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 8 *********************************************** *********************************************** Planning step 4/10 in round 9/10 Current state: | 0 5 0 Setting time for this decision to 13.7746s. THTS: Maximal search depth set to 7 Search time: 0.0175992s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 91.9687 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 84.1397 (in 2 real visits) roll(d3) : SOLVED with: 91.9687 (in 7 real visits) roll(d2) : SOLVED with: 70.9334 (in 7 real visits) roll(d1) : SOLVED with: 91.9687 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 8 *********************************************** *********************************************** Planning step 5/10 in round 9/10 Current state: | 0 5 4 Setting time for this decision to 14.6344s. THTS: Maximal search depth set to 6 Search time: 0.0169256s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 89.3333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85.25 (in 2 real visits) roll(d3) : SOLVED with: 80.1397 (in 7 real visits) roll(d2) : SOLVED with: 75.1397 (in 7 real visits) roll(d1) : SOLVED with: 89.3333 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 12 *********************************************** *********************************************** Planning step 6/10 in round 9/10 Current state: | 5 5 4 Setting time for this decision to 15.6089s. THTS: Maximal search depth set to 5 Search time: 0.0174841s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 85 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 85 (in 2 real visits) roll(d3) : SOLVED with: 82.25 (in 7 real visits) roll(d2) : SOLVED with: 78.25 (in 7 real visits) roll(d1) : SOLVED with: 78.25 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 7/10 in round 9/10 Current state: | 5 5 4 Setting time for this decision to 16.7225s. THTS: Maximal search depth set to 4 Search time: 0.0170985s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 68 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 68 (in 2 real visits) roll(d3) : SOLVED with: 65.375 (in 7 real visits) roll(d2) : SOLVED with: 62.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 8/10 in round 9/10 Current state: | 5 5 4 Setting time for this decision to 18.0075s. THTS: Maximal search depth set to 3 Search time: 0.0174845s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 51 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 51 (in 2 real visits) roll(d3) : SOLVED with: 48.75 (in 7 real visits) roll(d2) : SOLVED with: 46.75 (in 7 real visits) roll(d1) : SOLVED with: 46.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 9/10 in round 9/10 Current state: | 5 5 4 Setting time for this decision to 19.5066s. THTS: Maximal search depth set to 2 Search time: 0.017155s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 34 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 34 (in 2 real visits) roll(d3) : SOLVED with: 32.5 (in 7 real visits) roll(d2) : SOLVED with: 31.5 (in 7 real visits) roll(d1) : SOLVED with: 31.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** Planning step 10/10 in round 9/10 Current state: | 5 5 4 Setting time for this decision to 21.2784s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 86 Accumulated number of trials in root state: 1303437 Accumulated number of search nodes in root state: 4782131 Used RAM: 489572 Submitted action: noop() Immediate reward: 17 *********************************************** *********************************************** >>> END OF ROUND 9 -- REWARD RECEIVED: 119 *********************************************** *********************************************** >>> STARTING ROUND 10 -- REMAINING TIME 237s *********************************************** *********************************************** Planning step 1/10 in round 10/10 Current state: | 0 0 0 Setting time for this decision to 23.4044s. THTS: Maximal search depth set to 10 Search time: 0.0172385s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Action Selection: Exploitation in Root: 10 Exploration in Root: 9 Percentage Exploration in Root: 0.473684 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 110.224 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 98.0692 (in 2 real visits) roll(d3) : SOLVED with: 110.224 (in 7 real visits) roll(d2) : SOLVED with: 110.224 (in 7 real visits) roll(d1) : SOLVED with: 110.224 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 2/10 in round 10/10 Current state: | 0 0 0 Setting time for this decision to 26.0029s. THTS: Maximal search depth set to 9 Search time: 0.0170978s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 95.0692 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 83.2784 (in 2 real visits) roll(d3) : SOLVED with: 95.0692 (in 7 real visits) roll(d2) : SOLVED with: 95.0692 (in 7 real visits) roll(d1) : SOLVED with: 95.0692 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d1) Immediate reward: 3 *********************************************** *********************************************** Planning step 3/10 in round 10/10 Current state: | 4 0 0 Setting time for this decision to 29.2511s. THTS: Maximal search depth set to 8 Search time: 0.0171472s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 100.097 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 91.9687 (in 2 real visits) roll(d3) : SOLVED with: 100.097 (in 7 real visits) roll(d2) : SOLVED with: 100.097 (in 7 real visits) roll(d1) : SOLVED with: 84.2784 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d2) Immediate reward: 7 *********************************************** *********************************************** Planning step 4/10 in round 10/10 Current state: | 4 3 0 Setting time for this decision to 33.4273s. THTS: Maximal search depth set to 7 Search time: 0.0171995s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 92.6667 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 87.8333 (in 2 real visits) roll(d3) : SOLVED with: 92.6667 (in 7 real visits) roll(d2) : SOLVED with: 87.9687 (in 7 real visits) roll(d1) : SOLVED with: 82.1632 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 10 *********************************************** *********************************************** Planning step 5/10 in round 10/10 Current state: | 4 3 1 Setting time for this decision to 38.9955s. THTS: Maximal search depth set to 6 Search time: 0.0178702s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 78.8333 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 75.375 (in 2 real visits) roll(d3) : SOLVED with: 78.8333 (in 7 real visits) roll(d2) : SOLVED with: 75.3395 (in 7 real visits) roll(d1) : SOLVED with: 70.3812 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 11 *********************************************** *********************************************** Planning step 6/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 46.791s. THTS: Maximal search depth set to 5 Search time: 0.0173825s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 66.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 65.375 (in 2 real visits) roll(d3) : SOLVED with: 66.375 (in 7 real visits) roll(d2) : SOLVED with: 66.375 (in 7 real visits) roll(d1) : SOLVED with: 62.375 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 13 *********************************************** *********************************************** Planning step 7/10 in round 10/10 Current state: | 4 3 1 Setting time for this decision to 58.4843s. THTS: Maximal search depth set to 4 Search time: 0.0172081s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 50.375 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 47.75 (in 2 real visits) roll(d3) : SOLVED with: 50.375 (in 7 real visits) roll(d2) : SOLVED with: 46.9306 (in 7 real visits) roll(d1) : SOLVED with: 43.9306 (in 7 real visits) Used RAM: 489572 Submitted action: roll(d3) Immediate reward: 11 *********************************************** *********************************************** Planning step 8/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 77.973s. THTS: Maximal search depth set to 3 Search time: 0.0172128s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 19 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 39 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 39 (in 2 real visits) roll(d3) : SOLVED with: 38.75 (in 7 real visits) roll(d2) : SOLVED with: 38.75 (in 7 real visits) roll(d1) : SOLVED with: 36.75 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** Planning step 9/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 116.951s. THTS: Maximal search depth set to 2 Search time: 0.0171856s Statistics of THTS: Performed trials: 19 Created SearchNodes: 24 Cache Hits: 0 Skipped backups: 825323673 Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: Root Node: SOLVED with: 26 (in 23 real visits) Q-Value Estimates: noop() : SOLVED with: 26 (in 2 real visits) roll(d3) : SOLVED with: 25.5 (in 7 real visits) roll(d2) : SOLVED with: 25.5 (in 7 real visits) roll(d1) : SOLVED with: 24.5 (in 7 real visits) Used RAM: 489572 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** Planning step 10/10 in round 10/10 Current state: | 4 3 3 Setting time for this decision to 233.884s. THTS: Maximal search depth set to 1 Returning the optimal last action! Returning unique policy: noop() Statistics of THTS: Initializer: ExpandNode Heuristic weight: 1 Number of initial visits: 1 Heuristic: Statistics of DD Heuristic Seach[Steps: 10]: ROUND FINISHED Accumulated number of remaining steps in first solved root state: 96 Accumulated number of trials in root state: 1303456 Accumulated number of search nodes in root state: 4782155 Used RAM: 489572 Submitted action: noop() Immediate reward: 13 *********************************************** *********************************************** >>> END OF ROUND 10 -- REWARD RECEIVED: 97 *********************************************** *********************************************** Immediate rewards: Round 0: 3 3 5 10 11 11 15 16 16 16 = 106 Round 1: 3 3 3 7 8 13 17 17 17 17 = 105 Round 2: 3 6 7 12 13 14 12 15 15 15 = 112 Round 3: 3 7 7 8 9 12 15 15 15 15 = 106 Round 4: 3 8 13 15 18 18 18 18 18 18 = 147 Round 5: 3 7 9 10 13 14 13 12 16 16 = 113 Round 6: 3 5 7 8 12 11 12 15 13 18 = 104 Round 7: 3 6 6 10 11 13 15 14 16 16 = 110 Round 8: 3 3 8 8 12 17 17 17 17 17 = 119 Round 9: 3 3 7 10 11 13 11 13 13 13 = 97 >>> TOTAL REWARD: 1119 >>> AVERAGE REWARD: 111.9 *********************************************** PROST complete running time: 30.5469s