IVYXSTUDIO · IVY NODE

PII Unmask

ivy.node.pii-unmask · v0.1.0

ivyx✓

Puts the values pii-mask replaced back into any text or JSON value, such as a model's answer, and counts the placeholders restored. Placeholders the map does not hold, which a model may have made up, are left as they are and listed in unknown.

#pii#privacy#mask#text#json

Inputs

Field	Type	Description
valuerequired	Not declared	A text, object or list holding placeholders such as [PERSON_1].
mappingrequired	object	The mapping pii-mask returned.

Outputs

Field	Type	Description
valuerequired	Not declared	The same value with every known placeholder replaced by its value.
restoredrequired	integer	How many placeholders were replaced.
unknownrequired	array	Placeholders the map does not hold, each once.

Source

python

inp = __ivy_ctx__["nodes"][__ivy_node_id__]["input"]
import re

value = inp.get("value")
mapping = inp.get("mapping") or {}
if not isinstance(mapping, dict):
    raise ValueError("mapping must be the object pii-mask returned.")
PLACEHOLDER = re.compile(r"\[[A-Z]+_\d+\]")
restored, unknown = [0], []


def put_back(match):
    token = match.group(0)
    if token in mapping:
        restored[0] += 1
        return mapping[token]
    if token not in unknown:
        unknown.append(token)
    return token


def walk(v):
    if isinstance(v, str):
        return PLACEHOLDER.sub(put_back, v)
    if isinstance(v, list):
        return [walk(x) for x in v]
    if isinstance(v, dict):
        return {k: walk(x) for k, x in v.items()}
    return v


result = walk(value)

out = __ivy_ctx__["nodes"][__ivy_node_id__]["output"]
out["value"] = result
out["restored"] = restored[0]
out["unknown"] = unknown

Tests

Requires: python:3.9

round-trip
The masked statement line comes back whole.
model-answer
A model's JSON answer gets its values back at every depth, and a made-up placeholder is listed.
no-placeholders
A value with nothing to restore comes back unchanged.