Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix core docs related to linear constraints #2417

Closed
sgbaird opened this issue May 1, 2024 · 3 comments
Closed

Fix core docs related to linear constraints #2417

sgbaird opened this issue May 1, 2024 · 3 comments

Comments

@sgbaird
Copy link
Contributor

sgbaird commented May 1, 2024 •
edited
Loading

The constraint:

"2.0 * x3 + x1 + x2 >= 1"

produces:

ValueError: A linear constraint should be the form a*x + b*y - c*z <= d, where a,b,c,d are float constants and x,y,z are parameters. There should be no space in each term around the operator * while there should be a single space around each operator +, -, <= and >=.

However, the "core" docs imply a format like 1.0 * a + 0.5 * b <= 1.0.

image

If I change my constraint to:

"2.0*x3 + x1 + x2 >= 1"

it seems to be fine.

I suggest updating # 1.0 * a + 0.5 * b <= 1.0 to # 1.0*a + 0.5*b <= 1.0 (see #2418).

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], [line 37](vscode-notebook-cell:?execution_count=4&line=37)
     [15](vscode-notebook-cell:?execution_count=4&line=15) gs = GenerationStrategy(
     [16](vscode-notebook-cell:?execution_count=4&line=16)     steps=[
     [17](vscode-notebook-cell:?execution_count=4&line=17)         GenerationStep(
   (...)
     [32](vscode-notebook-cell:?execution_count=4&line=32)     ]
     [33](vscode-notebook-cell:?execution_count=4&line=33) )
     [35](vscode-notebook-cell:?execution_count=4&line=35) ax_client = AxClient(generation_strategy=gs, random_seed=42) # CHANGE: add random seed for reproducibility
---> [37](vscode-notebook-cell:?execution_count=4&line=37) ax_client.create_experiment(
     [38](vscode-notebook-cell:?execution_count=4&line=38)     parameters=[
     [39](vscode-notebook-cell:?execution_count=4&line=39)         {"name": "x1", "type": "range", "bounds": [0.0, total]}, # CHANGE: update parameter
     [40](vscode-notebook-cell:?execution_count=4&line=40)         {"name": "x2", "type": "range", "bounds": [0.0, total]}, # CHANGE: update parameter
     [41](vscode-notebook-cell:?execution_count=4&line=41)         {"name": "x3", "type": "range", "bounds": [0.0, total]}, # CHANGE: add new parameter
     [42](vscode-notebook-cell:?execution_count=4&line=42)         {"name": "x5", "type": "range", "bounds": [0.1, 10.0]}, # CHANGE: add new parameter
     [43](vscode-notebook-cell:?execution_count=4&line=43)     ],
     [44](vscode-notebook-cell:?execution_count=4&line=44)     objectives={
     [45](vscode-notebook-cell:?execution_count=4&line=45)         obj1_name: ObjectiveProperties(minimize=True),
     [46](vscode-notebook-cell:?execution_count=4&line=46)     },
     [47](vscode-notebook-cell:?execution_count=4&line=47)     parameter_constraints=[
     [48](vscode-notebook-cell:?execution_count=4&line=48)         f"x1 + x2 + x3 <= {total}", # CHANGE: update composition constraint
     [49](vscode-notebook-cell:?execution_count=4&line=49)         "x1 >= x2", # CHANGE: update order constraint
     [50](vscode-notebook-cell:?execution_count=4&line=50)         "x2 >= x3", # CHANGE: add order constraint
     [51](vscode-notebook-cell:?execution_count=4&line=51)         "2.0 * x3 + x1 + x2 >= 1", # CHANGE: add order constraint based on `x3 >= x4` and `x4 = 1 - (x1 + x2 + x3)`
     [52](vscode-notebook-cell:?execution_count=4&line=52)     ],
     [53](vscode-notebook-cell:?execution_count=4&line=53) )
     [55](vscode-notebook-cell:?execution_count=4&line=55) batch_size = 2
     [57](vscode-notebook-cell:?execution_count=4&line=57) for _ in range(10): # CHANGE: decrease number of iterations

File c:\Users\sterg\miniforge3\envs\honegumi\Lib\site-packages\ax\service\ax_client.py:338, in AxClient.create_experiment(self, parameters, name, description, owners, objectives, parameter_constraints, outcome_constraints, status_quo, overwrite_existing_experiment, experiment_type, tracking_metric_names, choose_generation_strategy_kwargs, support_intermediate_data, immutable_search_space_and_opt_config, is_test, metric_definitions)
    [333](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:333)     if len(objectives.keys()) > 1:
    [334](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:334)         objective_kwargs[
    [335](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:335)             "objective_thresholds"
    [336](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:336)         ] = self.build_objective_thresholds(objectives)
--> [338](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:338) experiment = self.make_experiment(
    [339](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:339)     name=name,
    [340](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:340)     description=description,
    [341](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:341)     owners=owners,
    [342](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:342)     parameters=parameters,
    [343](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:343)     parameter_constraints=parameter_constraints,
    [344](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:344)     outcome_constraints=outcome_constraints,
    [345](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:345)     status_quo=status_quo,
    [346](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:346)     experiment_type=experiment_type,
    [347](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:347)     tracking_metric_names=tracking_metric_names,
    [348](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:348)     metric_definitions=metric_definitions,
    [349](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:349)     support_intermediate_data=support_intermediate_data,
    [350](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:350)     immutable_search_space_and_opt_config=immutable_search_space_and_opt_config,
    [351](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:351)     is_test=is_test,
    [352](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:352)     **objective_kwargs,
    [353](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:353) )
    [354](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:354) self._set_runner(experiment=experiment)
    [355](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:355) self._set_experiment(
    [356](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:356)     experiment=experiment,
    [357](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:357)     overwrite_existing_experiment=overwrite_existing_experiment,
    [358](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/ax_client.py:358) )

File c:\Users\sterg\miniforge3\envs\honegumi\Lib\site-packages\ax\service\utils\instantiation.py:866, in InstantiationBase.make_experiment(cls, parameters, name, description, owners, parameter_constraints, outcome_constraints, status_quo, experiment_type, tracking_metric_names, metric_definitions, objectives, objective_thresholds, support_intermediate_data, immutable_search_space_and_opt_config, is_test)
    [860](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:860) if owners is not None:
    [861](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:861)     properties["owners"] = owners
    [863](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:863) return Experiment(
    [864](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:864)     name=name,
    [865](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:865)     description=description,
--> [866](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:866)     search_space=cls.make_search_space(parameters, parameter_constraints),
    [867](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:867)     optimization_config=optimization_config,
    [868](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:868)     status_quo=status_quo_arm,
    [869](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:869)     experiment_type=experiment_type,
    [870](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:870)     tracking_metrics=tracking_metrics,
    [871](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:871)     default_data_type=default_data_type,
    [872](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:872)     properties=properties,
    [873](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:873)     is_test=is_test,
    [874](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:874) )

File c:\Users\sterg\miniforge3\envs\honegumi\Lib\site-packages\ax\service\utils\instantiation.py:696, in InstantiationBase.make_search_space(cls, parameters, parameter_constraints)
    [692](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:692) search_space_cls = HierarchicalSearchSpace if is_hss else SearchSpace
    [694](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:694) parameter_map = {p.name: p for p in typed_parameters}
--> [696](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:696) typed_parameter_constraints = [
    [697](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:697)     cls.constraint_from_str(c, parameter_map) for c in parameter_constraints
    [698](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:698) ]
    [700](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:700) if any(
    [701](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:701)     any(
    [702](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:702)         isinstance(parameter_map[parameter], ChoiceParameter)
   (...)
    [705](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:705)     for constraint in typed_parameter_constraints
    [706](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:706) ):
    [707](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:707)     raise UnsupportedError(
    [708](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:708)         "Constraints on ChoiceParameters are not allowed. Try absorbing "
    [709](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:709)         "this constraint into the associated range parameter's bounds."
    [710](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:710)     )

File c:\Users\sterg\miniforge3\envs\honegumi\Lib\site-packages\ax\service\utils\instantiation.py:697, in <listcomp>(.0)
    [692](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:692) search_space_cls = HierarchicalSearchSpace if is_hss else SearchSpace
    [694](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:694) parameter_map = {p.name: p for p in typed_parameters}
    [696](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:696) typed_parameter_constraints = [
--> [697](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:697)     cls.constraint_from_str(c, parameter_map) for c in parameter_constraints
    [698](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:698) ]
    [700](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:700) if any(
    [701](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:701)     any(
    [702](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:702)         isinstance(parameter_map[parameter], ChoiceParameter)
   (...)
    [705](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:705)     for constraint in typed_parameter_constraints
    [706](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:706) ):
    [707](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:707)     raise UnsupportedError(
    [708](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:708)         "Constraints on ChoiceParameters are not allowed. Try absorbing "
    [709](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:709)         "this constraint into the associated range parameter's bounds."
    [710](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:710)     )

File c:\Users\sterg\miniforge3\envs\honegumi\Lib\site-packages\ax\service\utils\instantiation.py:399, in InstantiationBase.constraint_from_str(representation, parameters)
    [397](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:397) bound = float(tokens[-1])
    [398](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:398) if any(token[0] == "*" or token[-1] == "*" for token in tokens):
--> [399](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:399)     raise ValueError(
    [400](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:400)         "A linear constraint should be the form a*x + b*y - c*z <= d"
    [401](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:401)         ", where a,b,c,d are float constants and x,y,z are parameters. "
    [402](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:402)         "There should be no space in each term around the operator * while "
    [403](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:403)         "there should be a single space around each operator +, -, <= and >=."
    [404](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:404)     )
    [405](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:405) parameter_weight = {}
    [406](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:406) comparison_multiplier = (
    [407](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:407)     1.0 if COMPARISON_OPS[tokens[-2]] is ComparisonOp.LEQ else -1.0
    [408](file:///C:/Users/sterg/miniforge3/envs/honegumi/Lib/site-packages/ax/service/utils/instantiation.py:408) )

ValueError: A linear constraint should be the form a*x + b*y - c*z <= d, where a,b,c,d are float constants and x,y,z are parameters. There should be no space in each term around the operator * while there should be a single space around each operator +, -, <= and >=.
sgbaird added a commit to sgbaird/Ax that referenced this issue May 1, 2024
@sgbaird
Fix formatting of linear constraint (string version)
d5b8827 
closes facebook#2417
@sgbaird sgbaird mentioned this issue May 1, 2024
Closed
@eytan
Copy link
Contributor

eytan commented May 1, 2024 via email

@sgbaird
Copy link
Contributor Author

sgbaird commented May 2, 2024

Sorry, the point was that I did, but I thought it would help to have that spot in docs also reflect that.

@Balandat
Copy link
Contributor

Balandat commented May 2, 2024

It also seems like we should be able to parse the one with spaces just fine... Seems like we should be doing some better string normalization.

@Balandat Balandat mentioned this issue May 2, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix formatting of linear constraint (string version) sgbaird/Ax
3 participants
@eytan @Balandat @sgbaird