How to set up Random forest using Optuna
How to set up Random forest using Optuna
Since I wrote how to use ʻOptunalast time, I will describe the individual setting method from now on. .. There are various arguments that can be passed inRandomforest, but I set all the main ones in ʻOptuna.
I was wondering whether to pass max_depth, n_estimators as integers or numbers as categories, but this time I passed them as integers.
def objective(trial):
criterion = trial.suggest_categorical('criterion', ['mse', 'mae'])
bootstrap = trial.suggest_categorical('bootstrap',['True','False'])
max_depth = trial.suggest_int('max_depth', 1, 10000)
max_features = trial.suggest_categorical('max_features', ['auto', 'sqrt','log2'])
max_leaf_nodes = trial.suggest_int('max_leaf_nodes', 1, 10000)
n_estimators = trial.suggest_int('n_estimators', 30, 1000)
regr = RandomForestRegressor(bootstrap = bootstrap, criterion = criterion,
max_depth = max_depth, max_features = max_features,
max_leaf_nodes = max_leaf_nodes,n_estimators = n_estimators,n_jobs=2)
#regr.fit(X_train, y_train)
#y_pred = regr.predict(X_val)
#return r2_score(y_val, y_pred)
score = cross_val_score(regr, X_train, y_train, cv=5, scoring="r2")
r2_mean = score.mean()
return r2_mean
#Execute optuna and set hyperparameters
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=1000)
#Create an instance with tuned hyperparameters
optimised_rf = RandomForestRegressor(bootstrap = study.best_params['bootstrap'], criterion = study.best_params['criterion'],
max_depth = study.best_params['max_depth'], max_features = study.best_params['max_features'],
max_leaf_nodes = study.best_params['max_leaf_nodes'],n_estimators = study.best_params['n_estimators'],
n_jobs=2)
#learn
optimised_rf.fit(X_train ,y_train)
I used this to tune hyperparameters using a Boston dataset. It fits nicely.
Recommended Posts