table.bulk_fetch() with meta attributes?

Daniel Manocchio
Daniel Manocchio Member Posts: 24
10 Comments 5 Likes Photogenic
**

Hi Guys,

is there any way to have meta-attributes populated using the table.bulk_fetch() function without passing all their attribute definitions?

Calling bulk_fetch() with a list of (string) names only populates the attribute values, but not the meta-attribute values. Hence, I create a list of table attribute definitions containing both attributes and meta-attributes. When this list is used for calling bulk_fetch(), also the named meta-attributes are populated.

Now, I'm working on a project in which I generally need all the meta-attributes of a set of attributes. Building this list by table attribute definitiones is quite inconvenient. Hence I'm wondering whether there is a more elegant solution for this use case.

Ideas, anyone? :-)

Thanks in advance,
Daniel

Best Answer

  • Andy_Grigg
    Andy_Grigg Member, Employee Posts: 27
    5 Answers 10 Comments Name Dropper First Anniversary
    ✭✭✭✭
    Answer ✓

    Understood. I'll add the enhancement to the development backlog. We'll try to get to it (or something that satisfies the same use case) before the next release.

Answers

  • Andy_Grigg
    Andy_Grigg Member, Employee Posts: 27
    5 Answers 10 Comments Name Dropper First Anniversary
    ✭✭✭✭

    If you call Table.bulk_fetch() without any attributes specified, the scripting toolkit will fetch data for all attributes, meta-attributes, and pseudoattributes. If you need all attributes and meta-attributes, then this might be a good option.

    If not, we can look at an enhancement to be able to specify that all meta-attributes are included for the specified attributes in a bulk_fetch operation. This would probably be via an additional optional keyword argument on the bulk_fetch method.

    Please let me know what you think about the two options above.

  • Daniel Manocchio
    Daniel Manocchio Member Posts: 24
    10 Comments 5 Likes Photogenic
    **

    Hi Andy,
    thanks for your quick reply. Yeah, I've already used bulk_fetch() that way. However, for tables with a huge number of attributes and/or a higher number of records being processed, this becomes quite imperformant - at least on our system. So an additional argument like a boolean "include_metas=True" would be really great in some use cases :)

  • Daniel Manocchio
    Daniel Manocchio Member Posts: 24
    10 Comments 5 Likes Photogenic
    **

    Cool, thanks a lot! :)

  • Andy_Grigg
    Andy_Grigg Member, Employee Posts: 27
    5 Answers 10 Comments Name Dropper First Anniversary
    ✭✭✭✭

    Just to follow up on this, the solution we've chosen for the next release is to document the one or two lines of code necessary to add all meta-attributes to the bulk_fetch request. I don't want to go too much into the reasons here, but essentially it would add a lot of complexity to both the Scripting Toolkit itself and the bulk_fetch interface to do this properly.

    The documentation should be much clearer now though, and provides the code snippet required in a copy-and-paste-able form.

  • Daniel Manocchio
    Daniel Manocchio Member Posts: 24
    10 Comments 5 Likes Photogenic
    **

    Hi Andy,
    thanks for the update! I guess these "one or two lines of code" just extend the list of attribute definitions with the metas of each attribute? Anyway, I'm excited :smile:

  • Andy_Grigg
    Andy_Grigg Member, Employee Posts: 27
    5 Answers 10 Comments Name Dropper First Anniversary
    ✭✭✭✭

    That's exactly it, here's the snippet:

    table = db.get_table("MaterialUniverse")
    attribute_names = ["Density", "Price", "Young's modulus"]
    print("Fetch three attributes by name")
    table.bulk_fetch(records, attributes=attribute_names)
    
    attribute_definitions = [table.attributes[name] for name in attribute_names]
    print("Fetch three attributes by AttributeDefinition")
    table.bulk_fetch(records, attributes=attribute_definitions)
    
    attr_and_meta_definitions = attribute_definitions + [meta_attr
                                                         for attr in attribute_definitions
                                                         for meta_attr in attr.meta_attributes.values()]
    print("Fetch three attributes and their meta-attributes")
    table.bulk_fetch(records, attributes=attr_and_meta_definitions)
    
    attr_meta_and_pseudo_definitions = attr_and_meta_definitions + mpy.RecordProperties.all()
    print("Fetch three attributes, their meta-attributes, and all pseudo-attributes")
    table.bulk_fetch(records, attributes=attr_meta_and_pseudo_definitions)
    
  • Daniel Manocchio
    Daniel Manocchio Member Posts: 24
    10 Comments 5 Likes Photogenic
    **

    Hi Andy,
    ok then, seems like something that'll find its way as function in my personal toolbox :lol:

    Thanks & bests,
    Daniel